Node.js Native Development Introduction Complete Tutorial
I. About
This article is devoted to teaching you how to use Node.js to develop applications, which will teach you all the "advanced" JavaScript knowledge you need. It's enough for Node to get started with this article.
II. Code Status
All codes were tested by Chun Ge himself and passed correctly.
3. The Object of Reading Articles
1. Basic Programming
2. Technology enthusiasts who want to turn to Node.js backend
3.Node.js novice
IV. Getting to the Topic
1. Environmental Installation
Please move directly. Node.js official website As shown in the figure below, click on the latest version to download and install it.
After the installation of Node.js, open the terminal and enter the following commands at the terminal to check whether the installation is successful.
Last login: Tue Jun 27 09:19:38 on console liyuechun:~ yuechunli$ node -v v8.1.3 liyuechun:~ yuechunli$ npm -v 5.0.3 liyuechun:~ yuechunli$
If the versions of node and npm are displayed correctly, the installation of Node.js is successful.
2."Hello World"
The first mode of output
Okay, let's stop talking about nonsense and start our first Node.js application, "Hello World".
liyuechun:~ yuechunli$ node
> console.log("Hello World!");
Hello World!
undefined
> console. log ("from zero to one whole stack tribe!");
From zero to one whole stack tribe!
undefined
> process.exit()
liyuechun:~ yuechunli$ Enter the command node directly in the terminal, and then enter a console.log("Hello World!"); Enter and output Hello World.
Simply explain why an undefined appears after each print because when you enter the js code and press Enter, the node will output the return value after executing the code. If there is no return value, it will display undefined, which is similar to Chrome's debugging tool.
As shown in the code above, when you enter process.exit() and return, you exit node mode.
Second mode of output
Last login: Thu Jun 29 18:17:27 on ttys000 liyuechun:~ yuechunli$ ls Applications Downloads Pictures Creative Cloud Files Library Public Desktop Movies Documents Music liyuechun:~ yuechunli$ cd Desktop/ liyuechun:Desktop yuechunli$ mkdir nodejs Introduction liyuechun:Desktop yuechunli$ pwd /Users/liyuechun/Desktop liyuechun:Desktop yuechunli$ cd nodejs Introduction/ liyuechun:nodejs Introduction yuechunli$ pwd /Users/liyuechun/Desktop/nodejs Introduction liyuechun:nodejs Introduction yuechunli$ vi helloworld.js liyuechun:nodejs Introduction yuechunli$ cat helloworld.js console.log("Hello World!"); liyuechun:nodejs Introduction yuechunli$ node helloworld.js Hello World! liyuechun:nodejs Introduction yuechunli$
Order Interpretation:
ls: View the files and folders under the current path.
pwd: View the current path.
cd Desktop: Switch to desktop.
mkdir nodejs entry: Create the nodejs entry folder under the current path.
cd nodejs entry: enter the nodejs entry folder.
vi helloworld.js: Create a helloworld.js file and enter console.log("Hello World!") in the file to save and exit.
cat helloworld.js: View the contents of the helloworld.js file.
node helloworld.js: Execute the helloworld.js file under the current path.
PS: If you are not familiar with the command line, you can use other editors to create a helloworld.js file and enter console.log("Hello World! "), save the file to the desktop, then open the terminal, drag the helloworld.js file directly to the terminal, and execute node helloworld.js directly in the terminal to output Hello World!.
Well, I admit that the application is a bit boring, so let's do something dry.
Next we will pass VSCode Encoding Node.js.
5. A Complete web Application Based on Node.js
1. Use cases
Let's make the goal simple, but practical enough:
Users can use our applications through browsers.
When the user requests http://domain/start You can see a welcome page with a file upload form.
Users can select a picture and submit a form, and then the file will be uploaded to http://domain/upload When the page is uploaded, the picture will be displayed on the page.
Almost, you can go to Google now and find something to mess around with to complete the function. But let's not do it now.
Further, in order to achieve this goal, we need not only basic code, but also elegant code. We also need to abstract this to find a way to build more complex Node.js applications.
2. Analysis of different modules
Let's decompose the application. What parts do we need to implement in order to implement the use cases above?
We need to provide Web pages, so we need an HTTP server
For different requests, depending on the URL of the request, our server needs to respond differently, so we need a routing to correspond the request to the request handler.
When the request is received by the server and passed through the routing, it needs to be processed, so we need the final request handler.
Routing should also be able to process POST data, and encapsulate the data in a more friendly format and pass it to the request processing program, so the request data processing function is needed.
We need not only to process requests corresponding to URL s, but also to display content, which means that we need some view logic for request handlers to send content to users'browsers.
Finally, users need to upload pictures, so we need upload processing to handle the details.
Now let's start with the first part, the HTTP server.
6. Building Application Modules
1. A Basic HTTP Server
Create a server.js file with VSCode and save the file to the Noejs entry folder on the desktop.
Write the following in the server.js file:
let http = require("http"); http.createServer(function(request, response) { response.writeHead(200, {"Content-Type": "text/plain"}); response.write("Hello World"); response.end(); }).listen(8888);
The above code is a complete Node.js server. As shown in the figure below, click the left foot button of VSCode, open the VSCode terminal, and enter node server.js in the terminal to verify.
As shown in the figure above, a basic HTTP server is completed.
2. Analysis of HTTP Server Principle
In the case above, the first line requests the http module that comes with Node.js and assigns it to the http variable.
Next, we call the function provided by the HTTP module: createServer. This function returns an object with a method called listen, which has a numerical parameter specifying the port number that the HTTP server is listening on.
Let's ignore the function definition in parentheses of http.createServer for the moment.
We could have used this code to start the server and listen on port 8888:
var http = require("http"); var server = http.createServer(); server.listen(8888);
This code will only start a server that listens on port 8888 and does nothing else, not even respond to requests.
3. Function transfer
For example, you can do this:
Last login: Thu Jun 29 20:03:25 on ttys001 liyuechun:~ yuechunli$ node > function say(word) { ... console.log(word); ... } undefined > > function execute(someFunction, value) { ... someFunction(value); ... } undefined > > execute(say, "Hello"); Hello undefined >
Please read this code carefully! Here, we pass the say function as the first variable of the execute function. What is passed here is not the return value of say, but the say itself!
In this way, say becomes the local variable someFunction in execute, which can be used by calling someFunction() (in parenthesized form).
Of course, because say has a variable, execute can pass such a variable when calling someFunction.
We can, as we just did, pass a function as a variable by its name. But we don't have to go around the circle of "Define first, then pass". We can define and pass this function directly in parentheses of another function:
Last login: Thu Jun 29 20:04:35 on ttys001 liyuechun:~ yuechunli$ node > function execute(someFunction, value) { ... someFunction(value); ... } undefined > > execute(function(word){ console.log(word) }, "Hello"); Hello undefined >
Where execute accepts the first parameter, we directly define the function we are going to pass to execute.
In this way, we don't even need to name this function, which is why it is called an anonymous function.
This is our first intimate encounter with what I call "advanced" JavaScript, but we still have to go step by step. Now let's accept this: in JavaScript, one function can receive a parameter as another function. We can define a function first, then pass it, or we can define the function directly where the parameters are passed.
4. How does function passing make HTTP servers work
With this knowledge, let's take a look at our simple but not simple HTTP server:
var http = require("http"); http.createServer(function(request, response) { response.writeHead(200, {"Content-Type": "text/plain"}); response.write("Hello World"); response.end(); }).listen(8888); console.log("Please open it in the browser http://127.0.0.1:8888...");
Now it should look a lot clearer: we passed an anonymous function to the createServer function.
The same purpose can be achieved with such code:
/** * From zero to one whole stack tribe, add Erhiyidian Jian */ //Request the http module that comes with Node.js and assign it to the http variable. let http = require("http"); //Arrow function let onRequest = (request, response) => { response.writeHead(200, {"Content-Type": "text/plain"}); response.write("Hello World"); response.end(); } //Transfer functions as parameters http.createServer(onRequest).listen(8888); console.log("Please open it in the browser http://127.0.0.1:8888...");
Maybe now we should ask the question: Why do we use this way?
5. Event-driven callbacks
Event-driven is Node.js's native way of working, which is why it works so fast.
When we use the http.createServer method, of course, we don't just want a server that listens on a port, but also want it to do something when the server receives an HTTP request.
We created the server and passed a function to the method that created it. Whenever our server receives a request, this function is called.
This is the legendary callback. We pass a function to a method that calls it back when a corresponding event occurs.
Let's try the following code:
/** * From zero to one whole stack tribe, add Erhiyidian Jian */ //Request the http module that comes with Node.js and assign it to the http variable. let http = require("http"); //Arrow function let onRequest = (request, response) => { console.log("Request received."); response.writeHead(200, {"Content-Type": "text/plain;charset=utf-8"}); response.write("Add Wizard Wechat(ershiyidianjian),Join the whole stack tribe"); response.end(); } //Transfer functions as parameters http.createServer(onRequest).listen(8888); console.log("Server has started."); console.log("Please open it in the browser http://127.0.0.1:8888...");
In the figure above, when we execute the node server.js command, Server has started. Executes normally downward.
Let's see what happens when we open http://127.0.0.1:8888 in the browser.
You will find that when you open http://127.0.0.1:8888 in the browser, you will output Request received at the terminal, and the browser will output the phrase "ershiyidianjian" to join the whole stack of tribes.
Please note that when we visit a web page on the server, our server may output "Request received." twice. That's because most browsers are accessed by you. http://localhost:8888/ Trying to read http://localhost:8888/favicon... )
6. How does the server handle requests
Okay, let's briefly analyze the rest of our server code, which is the main part of our callback function onRequest().
When the callback starts and our onRequest() function is triggered, two parameters are passed in: request and response.
They are objects, and you can use their methods to process the details of HTTP requests and respond to requests (such as sending something back to the browser that made the request).
So our code is to use the response.writeHead() function to send an HTTP status 200 and the content-type of the HTTP header when the request is received, and use the response.write() function to send text in the corresponding body of HTTP to add an ershiyi Dianjian to join the whole stack tribe.
Finally, we call response.end() to complete the response.
For now, we don't care about the details of the request, so we don't use the request object.
7. Server-side modularization
What is a module?
let http = require("http"); ... http.createServer(...);
In the code above, Node.js comes with a module called "http", which we request in our code and assign the return value to a local variable.
This turns our local variable into an object with all the common methods provided by the http module.
It's a convention to give this local variable the same name as the module name, but you can do as you like:
var foo = require("http"); ... foo.createServer(...);
How to customize modules
Change the contents of the server.js file to the following.
/** * From zero to one whole stack tribe, add Erhiyidian Jian */ //Request the http module that comes with Node.js and assign it to the http variable. let http = require("http"); //Wrap up the previous content with a function let start = () => { //Arrow function let onRequest = (request, response) => { console.log("Request received."); response.writeHead(200, {"Content-Type": "text/plain;charset=utf-8"}); response.write("Add Wizard Wechat(ershiyidianjian),Join the whole stack tribe"); response.end(); } //Transfer functions as parameters http.createServer(onRequest).listen(8888); console.log("Server has started."); console.log("Please open it in the browser http://127.0.0.1:8888..."); } //Export `server'object, which contains a start function //The object format is /** * { * start * } */ //This object can be imported into other files and used with any name to receive it. exports.start = start;
Create a new index.js file under the current file path of server.js. The contents are as follows:
/** * From zero to one whole stack tribe, add Erhiyidian Jian */ //Importing server objects from `server'modules let server = require('./server'); //Start the server server.start();
Run the index.js file as shown below.
Everything works. In the case above, server.js is a custom module.
8. How to Route Requests
We provide the request URL and other required GET and POST parameters for the routing, and then the routing needs to execute the corresponding code based on these data (here "code" corresponds to the third part of the whole application: a series of processing procedures that actually work when the request is received).
Therefore, we need to look at the HTTP request and extract the request URL and GET/POST parameters. Whether this function should belong to routing or server (even as a module's own function) is really worth discussing, but here we tentatively define it as our HTTP server's function.
All the data we need will be included in the request object, which is passed as the first parameter of the onRequest() callback function. But to parse the data, we need additional Node.JS modules, url and querystring modules, respectively.
url.parse(string).query | url.parse(string).pathname | | | | | ------ ------------------- http://localhost:8888/start?foo=bar&hello=world --- ----- | | | | querystring(string)["foo"] | | querystring(string)["hello"]
Of course, we can also use the querystring module to parse the parameters in the POST request body, which will be demonstrated later.
Now let's add some logic to the onRequest() function to find the URL path of the browser request:
Next, I execute the node index.js command at the terminal, as follows:
bogon: How to route the request yuechunli$node index. JS Server has started. Please open http://127.0.0.1:8888 in the browser.
First, I open http://127.0.0.1:8888 in Safari browser. The browser shows the following results:
The effect of the console is as follows:
bogon: How to route the request yuechunli$node index. JS Server has started. Please open http://127.0.0.1:8888 in the browser. Request for / received.
Then I opened it in Google Browser http://127.0.0.1:8888... The browser effects are as follows:
The effect of the console is as follows:
Why do you print only one Request for / received... when making a request in Safari browser, and one more Request for /favicon.ico received... when accessing in Google browser, as shown in the figure above, because in Google browser, the browser's reason is to try to request favicon.ico icon.
In order to demonstrate the effect and not be disturbed by the favicon.ico request from Google Browser, I then request http://127.0.0.1:8888/start and http://127.0.0.1:8888/upload in Safari. Let's see what the console shows.
bogon: How to route the request yuechunli$node index. JS Server has started. Please open http://127.0.0.1:8888 in the browser. Request for /start received. Request for /upload received.
Well, our application can now differentiate requests by their URL paths -- allowing us to map requests to the handler using routing (not yet completed).
In the application we are building, this means that requests from / start and / upload can be processed using different code. Later we will see how these elements are integrated.
Now we can write the routing, create a file named router.js, and add the following:
/** * From zero to one whole stack tribe, add Erhiyidian Jian */ function route(pathname) { console.log("About to route a request for " + pathname); } exports.route = route;
As you can see, this code does nothing, but for now it should be. Before adding more logic, let's look at how to integrate routes and servers.
First, let's extend the server's start() function to pass the routing function as a parameter to the past:
/** * From zero to one whole stack tribe, add Erhiyidian Jian */ //Request the http module that comes with Node.js and assign it to the http variable. let http = require("http"); let url = require("url"); //Wrap up the previous content with a function let start = (route) => { //Arrow function let onRequest = (request, response) => { let pathname = url.parse(request.url).pathname; console.log("Request for " + pathname + " received."); route(pathname); response.writeHead(200, {"Content-Type": "text/plain;charset=utf-8"}); response.write("Add Wizard Wechat(ershiyidianjian),Join the whole stack tribe"); response.end(); } //Transfer functions as parameters http.createServer(onRequest).listen(8888); console.log("Server has started."); console.log("Please open it in the browser http://127.0.0.1:8888..."); } exports.start = start;
At the same time, we will expand index.js to allow routing functions to be injected into the server:
/** * From zero to one whole stack tribe, add Erhiyidian Jian */ //Importing server objects from `server'modules let server = require('./server'); let router = require("./router"); //Start the server server.start(router.route);
Here, the function we passed along still does nothing.
If you start the application now (node index.js, always remember this command line), and then request a URL, you will see the application output the corresponding information, which indicates that our HTTP server is already using the routing module, and will pass the requested path to the routing:
bogon: How to route requests v2.0 yuechunli$node index.js Server has started. Please open http://127.0.0.1:8888 in the browser. Request for / received. About to route a request for /
9. Routing to a real request handler
Now our HTTP server and request routing module can communicate with each other as we expected, just like a pair of intimate brothers.
Of course, this is not enough. Routing, as its name implies, means that we have to deal with different URL s in different ways. For example, the "business logic" of processing / start should be different from processing / upload.
In the current implementation, the routing process will "end" in the routing module, and the routing module is not really a "take action" module for requests. Otherwise, when our application becomes more complex, it will not be able to expand well.
For the time being, we call a function as a routing target a request handler. Now let's not rush to develop the routing module, because if the request processing program is not ready, how to improve the routing module will not make much sense.
Applications need new components, so adding new modules -- no longer surprising. Let's create a module called requestHandlers, add a placeholder function for each request handler, and then export these functions as a module method:
/** * From zero to one whole stack tribe, add Erhiyidian Jian */ function start() { console.log("Request handler 'start' was called."); } function upload() { console.log("Request handler 'upload' was called."); } exports.start = start; exports.upload = upload;
Now we pass a series of request handlers through an object, and we need to inject this object into the route() function in a loosely coupled way.
Let's first introduce this object into the main file index.js:
/** * From zero to one whole stack tribe, add Erhiyidian Jian */ //Importing server objects from `server'modules let server = require('./server'); let router = require("./router"); let requestHandlers = require("./requestHandlers"); //Object Construction var handle = {} handle["/"] = requestHandlers.start; handle["/start"] = requestHandlers.start; handle["/upload"] = requestHandlers.upload; //Start the server server.start(router.route, handle);
Although handles are not just a "thing" (a collection of request handlers), I recommend naming them with a verb, which allows us to use smoother expressions in routing, as will be explained later.
As you can see, mapping different URL s to the same request handler is easy: just add an attribute with the key'/'to the object, corresponding to requestHandlers.start, so that we can configure / start and'/' requests cleanly and concisely to the start handler.
After completing the definition of the object, we pass it to the server as an additional parameter. To this end, we modify server.js as follows:
/** * From zero to one whole stack tribe, add Erhiyidian Jian */ //Request the http module that comes with Node.js and assign it to the http variable. let http = require("http"); let url = require("url"); //Wrap up the previous content with a function let start = (route,handle) => { //Arrow function let onRequest = (request, response) => { let pathname = url.parse(request.url).pathname; console.log("Request for " + pathname + " received."); route(handle,pathname); response.writeHead(200, {"Content-Type": "text/plain;charset=utf-8"}); response.write("Add Wizard Wechat(ershiyidianjian),Join the whole stack tribe"); response.end(); } //Transfer functions as parameters http.createServer(onRequest).listen(8888); console.log("Server has started."); console.log("Please open it in the browser http://127.0.0.1:8888..."); } exports.start = start;
So we add the handle parameter to the start() function and pass the handle object as the first parameter to the route() callback function.
Then we modify the route() function in the route.js file accordingly:
With that, we're putting servers, routing, and request handlers together. Now we start the application and access http://127.0.0.1:8888/start in the browser. The following log shows that the system has invoked the correct request handler:
bogon: Routes to the real request handler yuechunli$node index. JS Server has started. Please open http://127.0.0.1:8888 in the browser. Request for /start received. About to route a request for /start Request handler 'start' was called.
And when you open http://127.0.0.1:8888/in the browser, you can see that the request is also processed by the start request handler:
bogon: Routes to the real request handler yuechunli$node index. JS Server has started. Please open http://127.0.0.1:8888 in the browser. Request for / received. About to route a request for / Request handler 'start' was called.
10. Let the request handler respond
Very good. But now it would be better if the request handler could return some meaningful information to the browser instead of just adding ershiyidian Jian to the whole stack of tribes.
It's important to remember that the browser's request for Erhiyidianjian and the whole stack of tribal information are still from the onRequest function in our server.js file.
In fact, "processing requests" means "responding to requests", so we need to enable request handlers to "talk" with browsers like onRequest functions.
11. Bad Ways to Realize
Modify the request Handler.js file as follows:
/** * From zero to one whole stack tribe, add Erhiyidian Jian */ function start() { console.log("Request handler 'start' was called."); return "Hello Start"; } function upload() { console.log("Request handler 'upload' was called."); return "Hello Upload"; } exports.start = start; exports.upload = upload;
Well. Similarly, request routing requires that the information returned to it by the request handler be returned to the server. Therefore, we need to modify router.js as follows:
/** * From zero to one whole stack tribe, add Erhiyidian Jian */ function route(handle, pathname) { console.log("About to route a request for " + pathname); if (typeof handle[pathname] === 'function') { return handle[pathname](); } else { console.log("No request handler found for " + pathname); return "404 Not found"; } } exports.route = route;
As shown in the above code, we also return some related error messages when the request cannot be routed.
Finally, we need to refactor our server.js to enable it to respond to the browser's content returned by the request handler through the request routing, as follows:
/** * From zero to one whole stack tribe, add Erhiyidian Jian */ //Request the http module that comes with Node.js and assign it to the http variable. let http = require("http"); let url = require("url"); //Wrap up the previous content with a function let start = (route,handle) => { //Arrow function let onRequest = (request, response) => { let pathname = url.parse(request.url).pathname; console.log("Request for " + pathname + " received."); route(handle,pathname); response.writeHead(200, {"Content-Type": "text/plain;charset=utf-8"}); var content = route(handle, pathname) response.write(content); response.end(); } //Transfer functions as parameters http.createServer(onRequest).listen(8888); console.log("Server has started."); } exports.start = start;
If we run the reconstructed application, everything will work well:
Request http://localhost:8888/start, and the browser will output Hello Start.
Requesting http://localhost:8888/upload will output Hello Upload.
The request http://localhost:8888/foo will output 404 Not found.
Okay, so what's the problem? Simply put, when there is a request handler that needs non-blocking operation in the future, our application will "hang".
No understanding? That's OK. Let's explain it in detail.
12. Blocking and non-blocking
Instead of explaining blocking and non-blocking here, let's modify the start request handler and let it wait 10 seconds before returning to Hello Start. Because there is no operation like sleep() in JavaScript, there is only a little Hack to simulate the implementation.
Let's modify requestHandlers.js to the following form:
/** * From zero to one whole stack tribe, add Erhiyidian Jian */ function start() { console.log("Request handler 'start' was called."); function sleep(milliSeconds) { var startTime = new Date().getTime(); while (new Date().getTime() < startTime + milliSeconds); } sleep(10000); return "Hello Start"; } function upload() { console.log("Request handler 'upload' was called."); return "Hello Upload"; } exports.start = start; exports.upload = upload;
In the above code, I call upload() first and return immediately as before. When the function start() is called, Node.js waits for 10 seconds before returning to "Hello Start". As shown in the following figure, waiting:
(Of course, this is just a simulation of 10 seconds of dormancy. In the actual scenario, there are many blocking operations, such as some long-term computational operations, etc.).
Next, let's see what changes have been brought about by our changes.
As usual, we need to restart the server first. To see the effect, we need to do some relatively complex operations (follow me): first, open two browser windows or tabs. Enter in the address bar of the first browser window http://localhost:8888/start But don't open it first!
Enter in the address bar of the second browser window http://localhost:8888/upload Likewise, don't open it first!
Next, do the following: in the first window ("/start") press Enter, then quickly switch to the second window ("/upload") press Enter.
Notice what happened: / start URL loading took 10 seconds, as we expected. However, / upload URL takes 10 seconds, and it doesn't have an operation like sleep() in the corresponding request handler!
Why on earth is that? The reason is that start() contains blocking operations. It blocked all other processing.
This is obviously a problem, because Node has always labelled itself as "everything executes in parallel except code in a node".
This means that Node.js can still process tasks in parallel without adding additional threads - Node.js is single-threaded. It implements parallel operations through event loop. To this end, we should make full use of this point - avoid blocking operations as much as possible, instead of using non-blocking operations.
However, to use non-blocking operations, we need to use callbacks by passing functions as parameters to other functions that take time to process (for example, hibernating for 10 seconds, querying databases, or doing a lot of calculations).
For Node.js, this is how it handles: "Hey, probable Expensive Function." You go on with your business. I (Node.js thread) won't wait for you. I'll go on with the code behind you. Please provide a callback Function (), and I'll call the callback when you're done with it. Number, thank you! ____________
(For more details on event polling, read Mixu's blog—— Understanding event polling for node.js. )
Next, we will introduce a wrong way to use non-blocking operations.
As we did last time, we exposed problems by modifying our applications.
This time, we'll use the start request processing program to "open up". Modify it to the following form:
/** * From zero to one whole stack tribe, add Erhiyidian Jian */ //We introduced a new Node.js module, child_process. It is used to implement a simple and practical non-blocking operation: exec(). var exec = require("child_process").exec; function start() { console.log("Request handler 'start' was called."); /** * exec()What did you do? * It executes a shell command from Node.js. * In this example, we use it to get all the files in the current directory ("ls-lah"). * Then, when the `start'URL request is made, the file information is output to the browser. * The following code is very intuitive: * A new variable content (initial value is "empty") is created. * Execute the "ls-lah" command, assign the result to content, and finally return the content. */ var content = "empty"; exec("ls -lah", function (error, stdout, stderr) { content = stdout; }); return content; } function upload() { console.log("Request handler 'upload' was called."); return "Hello Upload"; } exports.start = start; exports.upload = upload;
As usual, we start the server and then access it.“ http://localhost:8888/start" .
Load a beautiful web page with "empty" content. What's going on?
If you want to prove this, you can replace "ls-lah" with more time-consuming operations like "find/".
However, we are not satisfied with our non-blocking operations, are we?
Okay, next, let's fix this problem. In the process, let's first look at why the current approach doesn't work.
The problem is that exec() uses callback functions for non-blocking work.
In our example, the callback function is an anonymous function passed to exec() as the second parameter:
function (error, stdout, stderr) { content = stdout; }
Now we're at the root of the problem: our code is executed synchronously, which means that Node.js will execute the return content immediately after calling exec(); at this point, content is still "empty" because the callback function passed to exec() has not yet been executed -- because the operation of exec() is asynchronous.
Our "ls-lah" operation here is actually very fast (unless there are millions of files in the current directory). That's why callback functions can also be executed quickly -- but they're asynchronous anyway.
To make the effect more obvious, we imagine a more time-consuming command: "find /", which takes about a minute to execute on my machine. However, although in the request processing program, I replace "ls-lah" with "find /", when I open / start the URL, I still get an HTTP response immediately -- obviously, when exec() is executed in the background, Node.j. S itself will continue to execute the following code. And let's assume that the callback function passed to exec() will only be invoked after the "find /" command has been executed.
How can we display the list of files in the current directory to the user?
Okay, after understanding this bad implementation, let's show you how to get the request handler to respond to browser requests in the right way.
13. Request response with non-blocking operation
I just mentioned the phrase "the right way". In fact, usually the "right way" is not easy.
However, with Node.js, there is such a solution: function transfer. Now let's see how to achieve it.
So far, our application has been able to pass the content returned by the request handler (the content that the request handler will ultimately display to the user) to the HTTP server by passing values between the layers of the application (request handler - > request routing - > server).
Now we adopt the following new way of implementation: relative to the way of delivering content to the server, we use the way of "delivering" the server to the content this time. From a practical point of view, the response object (obtained from the server's callback function onRequest() is passed to the request handler through request routing. Subsequently, the handler can use functions on the object to respond to requests.
That's the way it works. Let's implement it step by step.
Start with server.js:
/** * From zero to one whole stack tribe, add Erhiyidian Jian */ //Request the http module that comes with Node.js and assign it to the http variable. let http = require("http"); let url = require("url"); //Wrap up the previous content with a function let start = (route,handle) => { //Arrow function let onRequest = (request, response) => { let pathname = url.parse(request.url).pathname; console.log("Request for " + pathname + " received."); route(handle, pathname, response); } //Transfer functions as parameters http.createServer(onRequest).listen(8888); console.log("Server has started."); } exports.start = start;
This time we pass the response object as the third parameter to the route() function, as opposed to the previous method of getting the return value from the route() function, and we remove all the functions related to the response from the onRequest() handler, because we want this part of the work to be done by the route() function.
Let's take a look at our router.js:
The same pattern: instead of retrieving the return value from the request handler earlier, this time the response object is passed directly.
If there is no corresponding request processor processing, we will directly return the "404" error.
Finally, we modify requestHandler.js as follows:
/** * From zero to one whole stack tribe, add Erhiyidian Jian */ //We introduced a new Node.js module, child_process. It is used to implement a simple and practical non-blocking operation: exec(). var exec = require("child_process").exec; function start(response) { console.log("Request handler 'start' was called."); exec("ls -lah", function (error, stdout, stderr) { response.writeHead(200, {"Content-Type": "text/plain"}); response.write(stdout); response.end(); }); } function upload(response) { console.log("Request handler 'upload' was called."); response.writeHead(200, {"Content-Type": "text/plain"}); response.write("Hello Upload"); response.end(); } exports.start = start; exports.upload = upload;
Our handler function needs to receive response parameters in order to respond directly to requests.
The start handler responds to requests in the anonymous callback function of exec(), while the upload handler is still a simple reply to "Hello World", only this time using the response object.
Then again we start the application (node index.js), and everything will work fine.
Open http:127.0.0.0:8888/start in the browser as follows:
Open http:127.0.0.0:8888/upload in the browser as follows:
If you want to prove that time-consuming operations in the / start handler do not block an immediate response to the / upload request, you can modify requestHandlers.js as follows:
var exec = require("child_process").exec; function start(response) { console.log("Request handler 'start' was called."); exec("find /", { timeout: 10000, maxBuffer: 20000*1024 }, function (error, stdout, stderr) { response.writeHead(200, {"Content-Type": "text/plain"}); response.write(stdout); response.end(); }); } function upload(response) { console.log("Request handler 'upload' was called."); response.writeHead(200, {"Content-Type": "text/plain"}); response.write("Hello Upload"); response.end(); } exports.start = start; exports.upload = upload;
In this way, when requesting http://localhost:8888/start, it takes 10 seconds to load, and when requesting http://localhost:8888/upload, it responds immediately, even though the/start response is still in process.
14. More useful scenarios
So far, we have done very well, but our application has no practical use.
Servers, request routing and request processing procedures have been completed. Let's add interaction to the website according to previous use cases: the user selects a file, uploads it, and then sees the uploaded file in the browser. To keep it simple, let's assume that the user will only upload the image, and then we apply it to display it in the browser.
Okay, let's do it step by step. Given that we've done a lot of introductions to the principles and techniques of JavaScript before, let's speed up this time.
To achieve this function, there are two steps: First, let's look at how to handle POST requests (non-file upload), and then we use an external module of Node.js for file upload. There are two reasons for adopting this approach.
First, although it's relatively simple to handle basic POST requests in Node.js, there's still a lot to learn in the process.
Second, using Node.js to process multipart POST requests is more complex, it is not in the scope of this article, but how to use external modules is within the scope of this book.
15. Processing POST requests
Consider a simple example: we display a text tarea for the user to enter content and then submit it it to the server through a POST request. Finally, the server receives the request and displays the input to the browser through the processing program.
/ The start request handler is used to generate a form with a text area, so we modified requestHandlers.js as follows:
/** * From zero to one whole stack tribe, add Erhiyidian Jian */ //We introduced a new Node.js module, child_process. It is used to implement a simple and practical non-blocking operation: exec(). var exec = require("child_process").exec; function start(response) { console.log("Request handler 'start' was called."); let body = '<html>'+ '<head>'+ '<meta http-equiv="Content-Type" content="text/html; '+ 'charset=UTF-8" />'+ '</head>'+ '<body>'+ '<form action="/upload" method="post">'+ '<textarea name="text" rows="5" cols="60"></textarea>'+ '<input type="submit" value="Submit text" />'+ '</form>'+ '</body>'+ '</html>'; response.writeHead(200, {"Content-Type": "text/html;charset=utf-8"}); response.write(body); response.end(); } function upload(response) { console.log("Request handler 'upload' was called."); response.writeHead(200, {"Content-Type": "text/plain;charset=utf-8"}); response.write("Hello Upload"); response.end(); } exports.start = start; exports.upload = upload;
The browser requests http://127.0.0.1:8888/start. The results are as follows:
For the rest of the article, let's explore a more interesting issue: when a user submits a form, the trigger / upload request handler handles POST requests.
Now that we are novice experts, it is natural to think of using asynchronous callbacks to achieve non-blocking processing of POST request data.
It is advisable to handle POST requests in a non-blocking manner because POST requests are generally "heavy" - users may enter a large amount of content. Processing requests with large amounts of data in a blocking manner will inevitably lead to blocking of user operations.
In order to make the whole process non-blocking, Node.js splits POST data into many small data blocks, which are then passed to the callback function by triggering specific events. Specific events here are data events (indicating the arrival of a new small block of data) and end events (indicating that all data has been received).
We need to tell Node.js which functions to call back when these events trigger. How to tell? We implement this by registering listener s on request objects. The request object here is passed to the onRequest callback function every time an HTTP request is received.
As follows:
request.addListener("data", function(chunk) { // called when a new chunk of data was received }); request.addListener("end", function() { // called when all chunks of data have been received });
The question arises. Where is this logic written? Now we just get the request object in the server -- we don't pass the request object to the request routing and request handler as we did before with the response object.
In my opinion, getting all the data from the request and then processing it to the application layer should be the task of the HTTP server. Therefore, I suggest that we process POST data directly in the server, and then pass the final data to the request routing and request processor for further processing.
Therefore, the idea is to put the callback function of data and end events directly in the server, collect all POST data in the callback of data events, when all data is received and the end event is triggered, the callback function calls the request routing and passes the data to it, and then the request routing passes the data to the request processing program.
What are we waiting for? We'll do it right away. Start with server.js:
/** * From zero to one whole stack tribe, add Erhiyidian Jian */ //Request the http module that comes with Node.js and assign it to the http variable. let http = require("http"); let url = require("url"); //Wrap up the previous content with a function let start = (route,handle) => { //Arrow function let onRequest = (request, response) => { let postData = ""; let pathname = url.parse(request.url).pathname; console.log("Request for " + pathname + " received."); request.setEncoding("utf8"); request.addListener("data", function(postDataChunk) { postData += postDataChunk; console.log("Received POST data chunk '"+ postDataChunk + "'."); }); request.addListener("end", function() { route(handle, pathname, response, postData); }); } //Transfer functions as parameters http.createServer(onRequest).listen(8888); console.log("Server has started."); } exports.start = start;
The code does three things: First, we set the encoding format for receiving data to UTF-8, then register the listener for "data" events to collect the new data blocks received each time and assign them to the postData variable. Finally, we move the call to request routing to the end event handler to ensure that it only touches when all data has been received. Send, and trigger only once. We also pass POST data to request routing because these data are used by request handlers.
The above code outputs the log when each data block arrives, which is not good for the final production environment (the amount of data may be large, remember?). But it's useful in the development phase to help us see what's happening.
I suggest that you try and try to enter a small piece of text, as well as a large piece of content. When a large piece of content, you will find that the data event triggers many times.
Cool again. Next, on the / upload page, we show the user's input. To achieve this function, we need to pass postData to the request handler and modify router.js as follows:
/** * From zero to one whole stack tribe, add Erhiyidian Jian */ function route(handle, pathname, response, postData) { console.log("About to route a request for " + pathname); if (typeof handle[pathname] === 'function') { handle[pathname](response, postData); } else { console.log("No request handler found for " + pathname); response.writeHead(404, {"Content-Type": "text/plain"}); response.write("404 Not found"); response.end(); } } exports.route = route;
Then, in requestHandlers.js, we include the data in the response to the upload request:
/** * From zero to one whole stack tribe, add Erhiyidian Jian */ //We introduced a new Node.js module, child_process. It is used to implement a simple and practical non-blocking operation: exec(). var exec = require("child_process").exec; function start(response, postData) { console.log("Request handler 'start' was called."); var body = '<html>'+ '<head>'+ '<meta http-equiv="Content-Type" content="text/html; '+ 'charset=UTF-8" />'+ '</head>'+ '<body>'+ '<form action="/upload" method="post">'+ '<textarea name="text" rows="20" cols="60"></textarea>'+ '<input type="submit" value="Submit text" />'+ '</form>'+ '</body>'+ '</html>'; response.writeHead(200, {"Content-Type": "text/html"}); response.write(body); response.end(); } function upload(response, postData) { console.log("Request handler 'upload' was called."); response.writeHead(200, {"Content-Type": "text/plain"}); response.write("You've sent: " + postData); response.end(); } exports.start = start; exports.upload = upload;
Well, we can now receive POST data and process it in the request handler.
The last thing we need to do is: at present, we are passing the whole message of the request to the request routing and request processing program. We should pass only the parts of POST data that we are interested in to request routing and request handlers. In our case, what we're interested in is actually the text field.
We can use the querystring module introduced earlier to achieve:
/** * From zero to one whole stack tribe, add Erhiyidian Jian */ //We introduced a new Node.js module, child_process. It is used to implement a simple and practical non-blocking operation: exec(). var exec = require("child_process").exec; var querystring = require("querystring"); function start(response, postData) { console.log("Request handler 'start' was called."); var body = '<html>'+ '<head>'+ '<meta http-equiv="Content-Type" content="text/html; '+ 'charset=UTF-8" />'+ '</head>'+ '<body>'+ '<form action="/upload" method="post">'+ '<textarea name="text" rows="20" cols="60"></textarea>'+ '<input type="submit" value="Submit text" />'+ '</form>'+ '</body>'+ '</html>'; response.writeHead(200, {"Content-Type": "text/html"}); response.write(body); response.end(); } function upload(response, postData) { console.log("Request handler 'upload' was called."); response.writeHead(200, {"Content-Type": "text/plain"}); response.write("You've sent the text: "+querystring.parse(postData).text); response.end(); } exports.start = start; exports.upload = upload;
Next, we visit http://127.0.0.1:8888/start in our browser, as shown in the following figure:
Click the Submit text button and jump to http://127.0.0.1:8888/upload. The effect is as follows:
Okay, that's the complete POST request.
15. Processing file upload
Finally, we implement our final use case: allowing users to upload images and display them in browsers.
We can learn two things through it:
How to install external Node.js module
And how to apply them to our applications
The external module we will use here is the node-formidable module developed by Felix Geisend rfer. It makes a good abstraction for parsing uploaded file data. In fact, to process file upload is to process POST data --- but the trouble is to deal with the details in detail, so it is more appropriate to adopt an off-the-shelf scheme here.
To use this module, you need to install it first. Node.js has its own package manager, called NPM. It makes it very convenient to install the external modules of Node.js.
First create the package.json file through npm init under the current project path:
PS: Enter npm init at the terminal and return all the way. The contents of the new package.json file are as follows:
{ "author" : "liyuechun", "description" : "", "license" : "ISC", "main" : "index.js", "name" : "fileupload", "scripts" : { "test" : "echo \"Error: no test specified\" && exit 1" }, "version" : "1.0.0" }
Next, enter the following command at the terminal to install the external module of formidable.
As follows:
liyuechun:fileupload yuechunli$ ls index.js requestHandlers.js server.js package.json router.js liyuechun:fileupload yuechunli$ npm install formidable npm notice created a lockfile as package-lock.json. You should commit this file. npm WARN fileupload@1.0.0 No description npm WARN fileupload@1.0.0 No repository field. + formidable@1.1.1 added 1 package in 1.117s liyuechun:fileupload yuechunli$
The package.json file changes as follows:
{ "author": "liyuechun", "description": "", "license": "ISC", "main": "index.js", "name": "fileupload", "scripts": { "test": "echo \"Error: no test specified\" && exit 1" }, "version": "1.0.0", "dependencies": { "formidable": "^1.1.1" } }
The overall change of the project is shown in the following figure:
Now we can use the formidable module, which is similar to the internal module by using the require statement.
let formidable = require("formidable");
What this module does here is submit the form through HTTP POST request, which can be parsed in Node.js. What we need to do is create a new IncomingForm, which is an abstract representation of the submitted form. Then we can use it to parse the request object and get the data fields needed in the form.
The official example of node-formidable shows how these two parts work together:
let formidable = require('formidable'), http = require('http'), util = require('util'); http.createServer(function(req, res) { if (req.url == '/upload' && req.method.toLowerCase() == 'post') { // parse a file upload let form = new formidable.IncomingForm(); form.parse(req, function(err, fields, files) { res.writeHead(200, {'content-type': 'text/plain'}); res.write('received upload:\n\n'); res.end(util.inspect({fields: fields, files: files})); }); return; } // show a file upload form res.writeHead(200, {'content-type': 'text/html'}); res.end( '<form action="/upload" enctype="multipart/form-data" '+ 'method="post">'+ '<input type="text" name="title"><br>'+ '<input type="file" name="upload" multiple="multiple"><br>'+ '<input type="submit" value="Upload">'+ '</form>' ); }).listen(8888);
If we save the above code into a file and execute it through node, we can submit a simple form, including file upload. Then, you can see the contents of the files object passed to the callback function by calling form.parse, as follows:
received upload: { fields: { title: 'Hello World' }, files: { upload: { size: 1558, path: './tmp/1c747974a27a6292743669e91f29350b', name: 'us-flag.png', type: 'image/png', lastModifiedDate: Tue, 21 Jun 2011 07:02:41 GMT, _writeStream: [Object], length: [Getter], filename: [Getter], mime: [Getter] } } }
In order to implement our functions, we need to apply the above code to our application. In addition, we need to consider how to display the contents of uploaded files (stored in. / tmp directory) in browsers.
Let's first solve the following problem: how can you see files stored on your local hard disk in a browser?
Obviously, we need to read this file into our server using a module called fs.
Let's add a request handler for the / show URL, which directly hardcodes the contents of the file. / tmp/test.png to the browser. Of course, first of all, you need to save the image to this location.
Modify requestHandlers.js to the following form:
/** * From zero to one whole stack tribe, add Erhiyidian Jian */ var querystring = require("querystring"), fs = require("fs"); function start(response, postData) { console.log("Request handler 'start' was called."); var body = '<html>'+ '<head>'+ '<meta http-equiv="Content-Type" '+ 'content="text/html; charset=UTF-8" />'+ '</head>'+ '<body>'+ '<form action="/upload" method="post">'+ '<textarea name="text" rows="20" cols="60"></textarea>'+ '<input type="submit" value="Submit text" />'+ '</form>'+ '</body>'+ '</html>'; response.writeHead(200, {"Content-Type": "text/html"}); response.write(body); response.end(); } function upload(response, postData) { console.log("Request handler 'upload' was called."); response.writeHead(200, {"Content-Type": "text/plain"}); response.write("You've sent the text: "+ querystring.parse(postData).text); response.end(); } function show(response, postData) { console.log("Request handler 'show' was called."); fs.readFile("./tmp/test.png", "binary", function(error, file) { if(error) { response.writeHead(500, {"Content-Type": "text/plain"}); response.write(error + "\n"); response.end(); } else { response.writeHead(200, {"Content-Type": "image/png"}); response.write(file, "binary"); response.end(); } }); } exports.start = start; exports.upload = upload; exports.show = show;
We also need to add this new request handler to the routing mapping table in index.js:
/** * From zero to one whole stack tribe, add Erhiyidian Jian */ //Importing server objects from `server'modules let server = require('./server'); let router = require("./router"); let requestHandlers = require("./requestHandlers"); //Object Construction var handle = {} handle["/"] = requestHandlers.start; handle["/start"] = requestHandlers.start; handle["/upload"] = requestHandlers.upload; handle["/show"] = requestHandlers.show; //Start the server server.start(router.route, handle);
After restarting the server, see the effect by visiting http://localhost:8888/show:
The reason is that there is No. / tmp/test.png picture under the current project path. We add a tmp folder under the current project path and drag a picture inside, named test.png.
Restart the server and visit http://localhost:8888/show to see the effect as follows:
Let's move on, starting with server.js -- removing the processing of postData and request.setEncoding (which node-formidable itself handles), and instead passing request objects to request routing:
/** * From zero to one whole stack tribe, add Erhiyidian Jian */ //Request the http module that comes with Node.js and assign it to the http variable. let http = require("http"); let url = require("url"); //Wrap up the previous content with a function let start = (route,handle) => { //Arrow function let onRequest = (request, response) => { let pathname = url.parse(request.url).pathname; console.log("Request for " + pathname + " received."); route(handle, pathname, response, request); } //Transfer functions as parameters http.createServer(onRequest).listen(8888); console.log("Server has started."); } exports.start = start;
Next comes router.js -- we no longer need to pass postData, this time we need to pass the request object:
/** * From zero to one whole stack tribe, add Erhiyidian Jian */ function route(handle, pathname, response, request) { console.log("About to route a request for " + pathname); if (typeof handle[pathname] === 'function') { handle[pathname](response, request); } else { console.log("No request handler found for " + pathname); response.writeHead(404, {"Content-Type": "text/html"}); response.write("404 Not found"); response.end(); } } exports.route = route;
Now the request object can be used in our upload request handler. node-formidable handles saving uploaded files to local / tmp directories, and what we need to do is make sure that the files are saved as. / tmp/test.png. Yes, we keep it simple and assume that only PNG images are allowed to be uploaded.
In this paper, fs.renameSync(path1,path2) is used to achieve. It is important to note that, as its name suggests, this method is executed synchronously, that is, if the renamed operation is time-consuming, it will block. Let's not consider this piece first.
Next, we put together the operations for file upload and renaming, as shown in requestHandlers.js below:
/** * From zero to one whole stack tribe, add Erhiyidian Jian */ var querystring = require("querystring"), fs = require("fs"), formidable = require("formidable"); function start(response) { console.log("Request handler 'start' was called."); var body = '<html>'+ '<head>'+ '<meta http-equiv="Content-Type" content="text/html; '+ 'charset=UTF-8" />'+ '</head>'+ '<body>'+ '<form action="/upload" enctype="multipart/form-data" '+ 'method="post">'+ '<input type="file" name="upload" multiple="multiple">'+ '<input type="submit" value="Upload file" />'+ '</form>'+ '</body>'+ '</html>'; response.writeHead(200, {"Content-Type": "text/html"}); response.write(body); response.end(); } function upload(response, request) { console.log("Request handler 'upload' was called."); var form = new formidable.IncomingForm(); console.log("about to parse"); form.parse(request, function(error, fields, files) { console.log("parsing done"); fs.renameSync(files.upload.path, "./tmp/test.png"); response.writeHead(200, {"Content-Type": "text/html"}); response.write("received image:<br/>"); response.write("<img src='/show' />"); response.end(); }); } function show(response) { console.log("Request handler 'show' was called."); fs.readFile("./tmp/test.png", "binary", function(error, file) { if(error) { response.writeHead(500, {"Content-Type": "text/plain"}); response.write(error + "\n"); response.end(); } else { response.writeHead(200, {"Content-Type": "image/png"}); response.write(file, "binary"); response.end(); } }); } exports.start = start; exports.upload = upload; exports.show = show;
Restart the server, browser access http://127.0.0.1:8888, the effect is as follows:
Select a picture to upload and see the effect:
16. Source Download
All source code: https://github.com/fullstacktribe/001nodejs