Structure and Development of Web Applications

Ports and Applications

Each computer has a series of communication points that are called ports. If a computer has a software process -- a program -- running on it and the program has control of a port and listens to the data sent to it, then the program can be communicated with. The communication follows a protocol, which lists the commands that the program understands.

The other party of the communication can reside on the same machine -- another program on the same machine may be the source of a message -- or the message can originate from a separate computer. For this to happen, the computer must be connected to a network, which allows computers to exchange data using a data link. The most prominent example of such a network is the internet, which has essentially become ubiquitous during the last decade.

Computers use IP addresses to identify other computers. In principle, each computer that is connected to the internet has an IP address that associates to it. An IP Address is a set of four numbers between 0 and 255 separated by a period (e.g. 127.0.0.1 -- which corresponds to your own computer). As there are only 232 possible options for the address, these are slowly running short due to the emergence of coffee pots with IP addresses and accompanying software -- the internet of things. A more recent version called IPv6 also exists; it has 2128 different address possibilities and essentially solves this issue.

Stepping up to the Task

From the programmer's point of view, communication between two computers is done using a socket. A socket is essentially a handle -- similar to a file handle -- that the programmer can use for both reading and writing. However, instead of working with a file, the "writing" is done to a given port at the remote machine, and "reading" is done from a stream of data that the remote machine sends.

Programming languages typically come with libraries which typically reduce the amount of code that a programmer must write to achieve a specific task. For example, accessing a remote computer in Java can be done using the Socket class from the Java API. In principle, the programmer only needs to create an instance of the Socket-class (the constructor takes the ip and the port of the target machine as parameters), and then -- for example -- read the data from the target machine.

String address = "127.0.0.1";
int port = 12321;

Socket socket = new Socket(address, port);
Scanner reader = new Scanner(socket.getInputStream());

The above example will -- depending on the target server -- throw an exception if the port cannot be connected to. See JavaTM Lesson on Exceptions for how to handle exceptions.

Port scanners are a common tool that security researchers use to identify services available on a given machine. Port scanners work by iterating through a range of ports, and attempting to connect to each of the ports. If a connection is successful, something has responded to the request, and it can be investigated further.

In this assignment, you will gain some hands-on experience on working with ports in Java and familiarize yourself with the tools used for some of the programming tasks in this course.

The assignment template that you can retrieve either using Test My Code (NetBeans plugin, experimental IntelliJ plugin) or the Test My Code web site contains the following code.

package sec.portscanner;

import java.net.Socket;
import java.util.Scanner;
import java.util.Set;
import java.util.TreeSet;

public class PortScanner {

    final static int MIN_PORT = 1024;
    final static int MAX_PORT = 49151;

    public static void main(String[] args) throws Exception {
        Scanner reader = new Scanner(System.in);

        System.out.println("Which address should I scan?");
        String address = reader.nextLine();
        System.out.println("Start at port?");
        int start = Integer.parseInt(reader.nextLine());
        System.out.println("End at port?");
        int end = Integer.parseInt(reader.nextLine());

        Set<Integer> ports = getAccessiblePorts(address, start, end);
        System.out.println("");

        if (ports.isEmpty()) {
            System.out.println("None found :(");
        } else {
            System.out.println("Found:");
            ports.stream().forEach(p -> System.out.println("\t" + p));
        }
    }

    public static Set<Integer> getAccessiblePorts(String address, int start, int end) {
        Set<Integer> accessiblePorts = new TreeSet<>();
        start = Math.max(start, MIN_PORT);
        end = Math.min(end, MAX_PORT);

        // write the code needed to scan the ports from
        // start to end at the given address

        return accessiblePorts;
    }
}

For this assignment, you should write the code needed for the method getAccessiblePorts to scan the given range of ports. The method should scan the ports at a given address, and then return the list of ports that have a service listening for them.

Once completed, submit your solution to the TMC server for assessment.

Note that while we implement port scanners and other tools, we will later also look into existing software that have similar functionality.

Note that port scanning can be seen as a preparation for a cyber attack. It may be also that your internet service provider (ISP) has prohibited making port scans. Before participating in the following quizz, verify that you are allowed to do so.

Talking with the remote program

If a computer has a port open, the program listening to that port can likely be talked with. One of the simplest approaches for trying out such discussion is the use of Telnet, which is available in most of the operating systems: if not, you can always download e.g. PuTTY. Telnet connections are made to a specific address and to a specific port. For example, a connection to the F-Secure web server could be initiated through the address f-secure.com and the port 80.

When discussing with an application, it is important to know the protocol -- discussion format -- that the application follows. One such example is the HTTP-protocol, which is used by web servers.

The basic command for retrieving the root content from a web-server is as follows.

GET / HTTP/1.1
Host: f-secure.com

In the above example, we first tell the server that we want to get the resource at "/", and that we are following version 1.1. of the HTTP. The next line describes the address we want to access -- this is entered as servers may host multiple web sites. An HTTP request is ended by two empty lines.

When launching telnet and retrieving the content from f-secure -site, we see something similar to the following.

username@machine:~$ telnet f-secure.com 80
Trying 104.126.172.25...
Connected to f-secure.com.
Escape character is '^]'.
GET / HTTP/1.1
Host: f-secure.com

HTTP/1.1 302 Moved Temporarily
Server: AkamaiGHost
Content-Length: 0
Location: http://f-secure.com/fi_FI/
...

In the above example, instead of returning the content of the page, the F-Secure web server asks us to look for the content from the address http://f-secure.com/fi_FI/.

Web Servers and Web Applications

Web servers are essentially programs that listen to incoming connections (typically, on port 80) and follow the HTTP-protocol. Whilst at first, web servers were mainly used to serve static content such as HTML pages and funny GIF images, they nowadays serve dynamic content that is often built separately for each user.

Web applications run on the web servers. Web servers listen for the incoming connections and forward the connections to the web applications. Developers who work with web applications very rarely implement functionality within the web servers.

Web applications include both client- and server-side functionality. Client-side functionality is executed in the browser of an user, i.e. you, whilst the server-side functionality is executed on the web server.

Whenever the user makes an action such as types in an URL and presses enter or clicks a link in the browser, the user's computer sends a request to the server. Once the server receives the request, it processes it and builds a corresponding response. The response may be, for example, HTML-code, JSON data, or an image that the browser should display to the user.

Flow of a typical request: (1) the user clicks a link on the browser, (2) the browser makes a request to the server, (3) the server receives the request and constructs a response, (4) the server returns the response, (5) the response is processed within the browser (and, e.g., shown to the user) -- not in the picture.

When constructing the functionality that is shown in the browser, development is typically focused on three separate and intertwined fronts. The structure of the view is constructed using HTML, the layout and theme using CSS, and possible dynamic functionality with JavaScript.

On the other hand, when constructing the functionality that is executed on the backend -- i.e. the server --, the developer typically focuses on the functionality that is needed to retrieve and construct the response that needs to be sent back to the user. This often involves connecting to other software such as a database server, and retrieving data from there. The server may -- again -- run either locally or on a separate machine. Naturally, the client- and server-side development can be interlinked, and software developers typically work on tasks that are related to both sides. That is, these days it is rather rare that a developer focuses solely on e.g. maintaining a database.

When the user accesses an online page using a browser, a request is sent to the server. The server creates the content of the response and sends it back to the user. If the content has links to other resources such as images or scripts, each of these resources are retrieved separately by the browser (except for the HTTP/2 Server Push-model).

Building a Simple Web Application

The main functionality of a web application is to create a response to each request. Developers do not typically implement the web-server functionality and the HTTP-protocol specifics, but use a framework that abstracts away many of the existing tasks. Here, we look at one such framework, called Spring.

Using Spring Boot, a web application that returns "Hello World!" to any request looks as follows.

import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.ResponseBody;

@Controller
public class HelloWorldController {

    @RequestMapping("*")
    @ResponseBody
    public String home() {
        return "Hello World!";
    }
}

There are a few items of interest. The @Controller annotation indicates that this class contains code that handles requests. The @RequestMapping annotation defines the request paths (e.g. "/hello") that the subsequent method will handle, and the @ResponseBody annotation essentially says that the response that has been created in the method should be sent directly to the user requesting the content.

In the above example, any request that is made to the web application will return the text "Hello World!".

In this assignment, you will familiarize yourself with the very basic functionality of the web framework, and will create an application that returns a "Hello Web!" response to the user.

The assignment template contains a few files. The file HelloWebApplication is used to launch the web application, and the file HelloWebController is used to create the response to the user. Modify the class HelloWebController so that the user will receive the text "Hello Web!" when he or she sends a request to the server.

Note that you may have to download the dependencies that the framework uses. In NetBeans, this can be done by selecting the Dependencies, and selecting "Download Declared Dependencies".

Once finished, submit the assignment to the TMC server.

Requests and Responses

Each request receives one response.

The annotation @RequestMapping defines the path or paths that the method following the annotation will handle. If the path is defined using an asterisk, i.e. @RequestMapping("*"), then all requests are forwarded to that specific method.

The path can be defined also more explicitly. For example, @RequestMapping("/secret") would forward all the requests that come to the path "/secret" to the method. In the example below, all such requests receive the response "Kryptos".

    @RequestMapping("/secret")
    @ResponseBody
    public String home() {
        return "Kryptos";
    }

Web applications typically respond to requests to multiple paths, where each path has specific functionality. In the example below, there are three separate paths, each of them returning a String as a response to the user.

import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.ResponseBody;

@Controller
public class PathController {

    @RequestMapping("/path")
    @ResponseBody
    public String path() {
        return "Path";
    }

    @RequestMapping("/route")
    @ResponseBody
    public String route() {
        return "Route";
    }

    @RequestMapping("/trail")
    @ResponseBody
    public String trail() {
        return "Trail";
    }
}

Each request may contain information that is being sent to the web application. In principle, there are two ways to handle this: (1) by adding parameters to the address, or by (2) adding parameters to the request body. We will look into the request body later in the course.

The parameters can be accessed using the @RequestParam-annotation. In the example below, the application greets every user that makes a request to the application; the application expects that the user sends information as a part of the request, and that the parameter name is "user".

import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.ResponseBody;

@Controller
public class GreetingController {

    @RequestMapping("/greet")
    @ResponseBody
    public String greet(@RequestParam String user) {
        return "Hi " + user + ", how are you?";
    }
}

If the server hosting the application would be running on port 8080, a request to http://localhost:8080/greet?user=Ada would receive a response "Hi Ada, how are you?".

In this assignment, you will familiarize yourself with handling (1) requests to several paths and (2) request parameters. Implement the following functionality to the class CalculatorController that can be found in the package sec.calculator.

  • A request to the path /add sums the values in the request parameters first and second together and returns the response to the user. Note that the parameters are numbers and should also be treated as such.
  • A request to the path /multiply multiplies the values in the request parameters first and second and returns the response to the user. Note that the parameters are numbers and should also be treated as such.

Once finished, submit the assignment to the TMC server.

Views to the User

The applications that we have worked on so far have received a request to a specific path and responded with a string. Whilst it is exactly the same end-to-end functionality that applications that send users HTML content use, HTML content is typically created using templates that include embedded commands that are used for determining the content that should be added to those templates. Here, we use a template engine called Thymeleaf.

Thymeleaf pages (the so called templates) reside in the project folder src/main/resources/templates or underneath it. You can find this folder in NetBeans when you click the folder "Other Sources".

In the example below, we have created an application that listens to the root path /. When the user makes a request to the application, a HTML page that has been created based on a template is returned to the user. The template that is used for creating the site is determined based on the string that the method returns -- here "index". This will lead to the framework looking for a template called index.html at src/main/resources/templates/. If the page is found (make sure the name is correct!), Thymeleaf will handle the page and return it to the user.

import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;

@Controller
public class ThymeleafController {

    @RequestMapping("/")
    public String home() {
        return "index";
    }
}

Note that the method that handles the request no longer has the annotation @ResponseBody. Effectively this means that we do not wish that the response from the method is sent directly to the user, but that it is used to determine the template that will be used to create the view.

Next, we will look into returning content created using a template. Implement the following functionality to the class HelloTemplateController (in the package sec.hellotemplates):

  • A request to the path / will return a response based on the template index.html that resides in src/main/resources/templates/.
  • A request to the path /video will return a response based on the template video.html that resides in src/main/resources/templates/.

Once finished, submit the assignment to the TMC server.

Adding data to the view

Adding data to the view is done using a Model-object. The Model object is automatically taken into use when it is added as a parameter to a method that handles incoming requests.

import org.springframework.stereotype.Controller;
import org.springframework.ui.Model;
import org.springframework.web.bind.annotation.RequestMapping;

@Controller
public class TemplatesAndDataController {

    @RequestMapping("/")
    public String home(Model model) {
        return "index";
    }
}

The Model is essentially a map with strings as keys and any objects as values. In the example below, we add the value "Hello World!" to the model with a key "text".

import org.springframework.stereotype.Controller;
import org.springframework.ui.Model;
import org.springframework.web.bind.annotation.RequestMapping;

@Controller
public class TemplatesAndDataController {

    @RequestMapping("/")
    public String home(Model model) {
        model.addAttribute("text", "Hello World!");
        return "index";
    }
}

When a request is made to the above method, the Model object and the data in it will be available for the template. If the template would be as follows, the value from the model that corresponds to the key "text" would be added to the element with the th:text="${text}"-attribute.

<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" xmlns:th="http://www.thymeleaf.org">
    <head>
        <title>Title</title>
    </head>

    <body>
        <h1>Something nice!</h1>

        <h2 th:text="${text}">Testing..</h2>
    </body>
</html>

Forming up: Content from Forms

Web applications may contain forms that are used to send content to the application. Forms are defined in HTML (see form) using the form-element. The form-element will contain the path to which the content will be sent to, the type of the request, and the data. For now, the type of the request will be POST.

The data is defined using fields such as the input field (<input type="text"...), and the content is sent to the server using a button (<input type="submit"...). The form below is submitted to the root path of the application, and the field that the user can input content to has a name "content".

<form th:action="@{/}" method="POST">
    <input type="text" name="content"/>
    <input type="submit"/>
</form>

Handling Lists

One may also include lists to the model. In the example below, we retrieve a request parameter called "content", and add it to the list. The parameter has been defined as non mandatory using the @RequestMapping annotation attribute required = false.

import java.util.List;
import java.util.ArrayList;
import org.springframework.stereotype.Controller;
import org.springframework.ui.Model;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;

@Controller
public class ListController {
    private List<String> data;

    public ListController() {
        this.data = new ArrayList<>();
        this.data.add("Hello world!");
    }

    @RequestMapping(value = "/")
    public String home(Model model, @RequestParam(required = false) String content) {
        if(content != null && !content.trim().isEmpty()) {
            this.data.add(content);
        }

        model.addAttribute("list", data);
        return "index";
    }
}

Iterating a list in the Thymeleaf template can be done using the attribute th:each. The basic syntax for th:each is as follows.

<p th:each="item : ${list}">
    <span th:text="${item}">hello world!</span>
</p>

In the example above, we search for a value with the key list and print all the items in the list one by one. The attribute th:each can be used in principle with any repeatable element; it could be used, for example, with the li-element as follows.

<ul>
    <li th:each="item : ${list}">
        <span th:text="${item}">hello world!</span>
    </li>
</ul>

Note! One of the most classic mistakes is to define the list of items as a string th:each="item : list". This does not work.

In this assignment we will look into using lists and using them as data for the templates.

The assignment has the server side functionality that handles a request to the root path and adds a list for processing for Thymeleaf. The template that is used to create the site is missing some functionality however.

Your task is to (1) print the values in the list using the th:each-attribute that we saw previously, and (2) add a form that can be used to send content to the server.

Once finished, submit the assignment to the TMC server.

Implement a notebook application to the class NotebookController in the package sec.notebook and the template index.html. The notebook application should list the existing notes and allow adding new notes. If there are more than 10 notes, the application should only show the ten most recent ones.

Once finished, submit the assignment to the TMC server.

During this part of the securing software course, we have taken the first steps into understanding web applications. The next part will look into using databases and the underlying HTTP protocol.

Table of contents

Some parts of this page might not work on your current browser. Consider switching to either Chrome or Firefox. Got it!