8. Communicating with a Backend

Previously, we learned about DOM events and using them to make our website interactive. Now let's make our website into a web application by allowing it to communicate with a server (called the backend). There are two ways to do this: the old way using HTML forms and the new way using AJAX (Asynchronous JavaScript).

8.1 Review of HTML Forms

For historical context we briefly review forms.

A bit of history. Forms used to be the only way for a user of a website to communicate with the server. Essentially, they are an HTML-only way to send data to the server. Let's look at an example of a web page that uses forms. Can you guess what the below code does when the user clicks the "submit my info" button?

<form action="example.com/hello" method="POST">
    <input type="text" name="my_name" />
    <button type="submit">submit my info</button>
</form>

If you guessed that the web browser would send the contents of the form input elements to the server, you guessed right! In particular, the data that gets sent to the server looks like a dictionary with a single key and value:

{ my_name: "<whatever the user inputted into the single text input element>" }

Notice that the single key - my_name - corresponds exactly to the name attribute of the html text input element.

Now, some things stand out to us about the behavior of the web page after the user clicks submit:

  • When the user receives the server's response, the entire web page will reload and interpret the server's response as a new HTML page! For this reason, when we use form elements to send data to the server, the server should always be returning a completely new HTML page, rather than chunks of data, because the web browser has this default behavior of interpreting the response as a new page.
  • There is very little we can do to change how the web page sends data to the server. That is, it will by default send data in this prescribed way of sending a dictionary with keys corresponding to the input element's name attributes, and values corresponding to the respective input element's values.
  • The good thing is that we generally don't need to change this behavior, and in the olden days before web applications (when web sites were literally just HTML) this was all people needed. We still teach the form way of doing things because of how simple it is.

8.2 Asynchronous JavaScript

Sometime in the mid 2000s, web applications became much more complex, and interactions with the server such as posting comments and receiving notifications in real time became much more common; since hundreds of these interactions could occur in the lifetime of a page, reloading the page for every interaction became unnecessary and infeasible. Instead, web pages used raw HTTP requests to request small chunks of data from web APIs and iteratively modify the page instead of reloading the page for every server interaction. This mode of interaction is historically called AJAX ("Asynchronous JavaScript").

8.2.1 Making Asynchronous HTTP Requests

The Fetch API is present in every modern browser and provides a simple interface to make asynchronous HTTP requests using JavaScript.

8.2.1.1 GET Fetch Requests

Making GET requests is very easy using the Fetch API; here is an example script that gets the weather from the Weather Underground API:

fetch("https://api.wunderground.com/api/MY_API_KEY/conditions/q/CA/San_Francisco.json")
  .then(function(response) {
    return response.json();
  })
  .then(function(json) {
    console.log(json.current_observation.weather);
  });

Notice the use of .then(function) - this is because the fetch API makes use of promises, which are essentially a cleaner way of adding event listener functions for asynchronous events. We do this because receiving a response to our HTTP request is an asynchronous event; we don't know when or if it will even happen, so the best we can do is listen for the event in the background while the rest of our code continues to execute.

Furthermore, fetch requests do not send authentication data such as cookies by default; in order to include cookies with our request, we must set the request property "credentials" in our second argument:

fetch("https://api.wunderground.com/api/MY_API_KEY/conditions/q/CA/San_Francisco.json", {
  credentials: "include"
});
8.2.1.2 POST Fetch Requests

Making POST requests requires setting more request properties in our second argument. Here is an example POST request:

var postData = new FormData();
postData.append("username", "dirks");
fetch("example.com/hello", {
  method: "POST",
  body: postData,
}).then(function(response) {
  return response.json();
}).then(function(json) {
  // ...
  // do whatever we want with the response data
  // ...
})

In the interest of replacing clunky forms with fetch requests, here is the same toy example above we made using HTML forms, but using fetch requests instead of forms:

HTML:

<input type="text" id="name_input" />
<button id="submit_btn">submit my info</button>

JS:

document.getElementById("submit_btn").addEventListener("click", function(e) {
  var textInput = document.getElementById("name_input").value;
  var formData = new FormData();
  formData.append("my_name", textInput);
  fetch("example.com/hello", {
    method: "POST",
    body: formData,
  });
});

Now, the HTML form used to decide everything for us, including:

  • What data to send to the server, since forms automatically grab the values of all input elements in between the form tags.
  • How to send that data, as forms always send data as an object with keys corresponding to input element name attributes.

We're no longer using forms (Notice the lack of <form> tags! Luckily, we can still use <input> elements outside of forms), so we have to decide these things ourselves. In particular, we have to explicitly grab the value of the input element ourselves, name the HTTP method being used, and set the request body to a FormData object with a single key "my_name". This is a bit of extra work we have to do, but in return, we get far more flexibility. And of course, the best part is that our page no longer automatically reloads and can do these requests asynchronously.

A Note on POST Data Formatting

The data send in the body of a POST request can be formatted in several ways, as specified by the content-type header in the request properties. There are 3 important content types:

  • multipart/form-data, where the body should be a FormData object.
  • application/x-www-form-urlencoded, where the body should be a string of key-value pairs in url encoded format.
  • application/json, where the body should be a stringified JavaScript object, e.g. JSON.stringify({ username: "dirks" }).

When using HTML forms, the POST request and its data is always automatically formatted according to content-type: multipart/form-data or content-type: application/x-www-form-urlencoded; a fetch request sending FormData will also automatically be formatted according to content-type: multipart/form-data. The latter format supports sending files, and both formats will be interpreted as form submissions by most backend frameworks - in particular, Django can access form fields of these two content formats using request.POST.

On the other hand, some backend frameworks do not support parsing the (often complicated) form content types. A typical Express.js setup, for instance, requires additional middleware for form data, but will process content-type: application/json just fine. Almost all modern APIs use JSON to communicate, so for this reason application/json should be used as a default POST content type.

8.2.1.3 Catching Fetch Errors

Occasionally our fetch requests will fail to be fulfilled with a response or will receive an HTTP error, and our response listener will never be called. We can catch these errors easily by simply adding an error listener to the promise:

fetch("https://api.wunderground.com/api/MY_API_KEY/conditions/q/CA/San_Francisco.json")
  .then(function(response) {
    return response.json();
  })
  .then(function(json) {
    console.log(json.current_observation.weather);
  })
  .catch(function(error) {
    console.error("Error:", error);
  });

Thus using fetch, we can asynchronously do everything that an HTML form can do and more, including send input data and files. Read the Fetch documentation for more.

8.3 Real-Time Communication

Modern web applications now not only support client-server communications without reloading the page, but also near real-time communications - consider the example of a chat app like Slack or Facebook Messenger, where users can send and receive messages with little to no delay. These web apps use asynchronous JavaScript to implement real-time communication as well.

8.3.1 Long Polling

The traditional way to implement real-time communications or push notifications in the web browser is through a technique known as long polling. In long polling, the client sends a pending HTTP request to some "pull" endpoint on the server. If the server has no new information for the client when the poll request is received, instead of sending an empty response, the server holds the request open and does not issue an HTTP response until response information becomes available. On receiving the server's response, the client immediately issues another pending HTTP request and continues the cycle forever. In code:

function pollNewMessages() {
  fetch("example.com/pull").then(function(res) {
    // handle new messages...
    pollNewMessages();
  });
}

pollNewMessages();

Facebook Messenger is known to use this technique. The main advantage of long polling is that it is supported by all browsers, unlike more advanced technologies and protocols.

8.3.2 Websockets

Websockets is a more recent technology that implements real-time communication in a more efficient, bi-directional protocol separate from HTTP. Under the hood, Websockets also uses polling, but it is free to send smaller ping and pong packets as it does not need to include extraneous information required with HTTP requests. With Websockets, both the server and client must support the Websockets protocol; luckily, support across modern browsers is good.

Read more about using Websockets in the browser here. Additionally, you can read more about the advantages and disadvantages of long polling vs. Websockets here.

results matching ""

    No results matching ""