It is well known that HTTP is the protocol of the Web, what is probably less known by many people is that the base idea of HTTP is indeed quite simple and can be used for many other applications even not related with internet.
An example of such application and very useful nowadays would be communicating devices that are attached to a home WLAN. For instance a PC with a smart phone, a tablet etc.
In this article we show how to do it and for that we will implement some basic but quite demonstrative functionalities like getting a screenshot from the PC, interchanging files (download and upload) and save notes into the PC written from a device.
The demos use a convenient and tiny HTTP server but they differ from typical Web applications in several ways. Thought to be ran in a secured network like a home WLAN there is no need to deal with security issues. Also because of this we can make the server play any role we want thus having full control of it which can not the case in a usual web application.
Http very briefly
Although the HTTP specification is quite long and contain a lot of rules and features we only need to know a small but fundamental part of it. Partly because many things like transferring files are already implemented in the server and client that we are going to use and partly because in "home" applications like these there is more freedom and flexibility for the design than in a generic product thought to have many users. Finally it is probably just the basis of this protocol, its simplicity and inherent flexibility the most powerful part of HTTP.
TCP/IP base communication
Http has on its base TCP/IP which is a point to point and a client-server communication protocol. Each point is identified by an IP address which is unique for each device (i.e a PC, smart phone etc) in a network (i.e. home WLAN or internet) and a port number where each device has more than 65000 port numbers available.
For example two TCP/IP points in a home WLAN could be
Device IP TCP port
----------- --------------- ----------
PC 192.168.178.22 8080
Smart phone 192.168.178.36 1727
In TCP/IP client-server means the server is listening to its port for incoming requests from clients, when a request is received it comes along with the IP and port number of the client, so the server can respond to the request. By now think of REQUEST and RESPONSE just as a set of an arbitrary number of bytes.
(PC) (Smart phone)
<------------- REQUEST ------------|
------------- RESPONSE ----------->|
This request-response principle is simple and flexible but also has some restrictions that has to be taken into account when designing an application using it. Among other things, what we can say about this kind of communication is
- A server has to be started first and be awake (listening) always
- A client needs to know the IP and port of the server to communicate with it.
- The communication is started by the client with a request and the server can just respond to that request.
- The server does know anything about clients until it receives requests from them.
The TCP/IP address (IP and port) is not shareable, which means we cannot have two applications located in the same machine (same IP address) opening the same port for HTTP communication. Still multiple clients, each one having its own unique TCP/IP address, might send requests to the same server simultaneously.
Above the TCP/IP protocol HTTP defines how the REQUEST and RESPONSE has to be formed. For that it uses a simple structure called HTTP message which may have up to three parts separated by CRLF (carriage return and line feed characters): START line, HEADER part and BODY part.
General Syntax Request example Response Example
----------------- ----------------------------- ------------------------------
START LINE<CRLF> GET /myIndex.html HTTP/1.1 HTTP/1.1 200 OK
MESSAGE HEADER<CRLF> Content-Type: text/html
... Content-Length: 170
MESSAGE HEADER<CRLF> ...
MESSAGE BODY <html><body> ...
All these parts can be used to send information from client to server and viceversa. Let's take a closer look at them.
The start line is different for REQUEST and RESPONSE messages. In the REQUEST the start line has the function of telling the server what we want to do: get a resource, store something etc and for that giving some kind of detail is needed, like a path of a file, an id or more things. The general syntax is
METHOD URI HTTP_VERSION
METHOD : GET, POST, ...
URI : path or path?query
HTTP_VERSION : HTTP/1.1
For METHOD there are more defined values in the specification but in many cases just GET and POST are used as we will do here.
"path" might correspond or not to a physical file in the server. In many cases of GET requests it does but in the end it is the server who decides the meaning of "path" and in general of the whole request. For instance, taking together METHOD and "path" it is possible to describe exactly the same actions using more or less number of methods. Just as an example we could express the operations of retrieving, storing and removing an event as
using GET,PUT,DELETE using only POST
GET /event POST /get/event
PUT /event POST /store/event
DELETE /event POST /delete/event
In general the URI in an HTTP message does not need host and port information as in a URL in the browser because these values are already known by the TCP/IP connection itself. Some examples showing this
Typical URL in browser Request received by the server
since the browser always performs a GET in these cases
Finally "query" is an optional string with "variable=value" units separated by the character '&'. This is a good place to put some parameters to complete the request information. An example of the query part
In the RESPONSE the start line is
HTTP_VERSION STATUS_ID STATUS_TEXT
it contains much less information since only a set of predefined status can be returned, for example "200 OK" or the well known "404 Not Found". We will use only
HTTP/1.1 200 OK
any other feedback to the client still can be sent by using response headers or body.
The header part is composed for an arbitrary number of lines each one consisting on two strings separated by a colon, the first string is the header name and the second its value.
In a HTTP application custom headers can be set from client and server side to pass information. There are as well standard headers defined in the specification, in particular one is not just important but critical for the protocol itself: the Content-Length. Its value has to be exactly the number of bytes of the body (if any) in order for the communication to work. Fortunatelly, both server and client (browser) will fill this header automatically, so we don't have to worry about it.
Also automatically the header "Content-Type" will be set in requests and responses, this is needed for instance by the browser in order to treat properly css, html etc contents.
Other headers even standard ones send from browsers can be ignored as they are irrelevant for the applications, although they could be handled if desired.
Http client and servers
A communication application using HTTP can be implemented in almost any language. Nowadays it makes a lot of sense to use modern browsers for the client side. They have tons of features and are already installed on most devices. Of course by the use of browsers we have to accept the limitations imposed for security reasons on the access and control of the host device itself, but for many application purposes it should not be an issue.
For the server side we will use an open source written by myself called gastona (gastona.jar) which is general purpose application builder and can implement an http server in a very straighforward way.
HTTP with browsers
There are basically two requests, actually the most frequently used and the ones that we are going to use in this demo, GET and POST.
The GET method is used to retrieve resources that in general are going to be rendered or used for rendering purposes in the browser. For example, when we click on a link in a html page, the content (attribute href of "a" tag) is retrieved using a GET request. The URI to retrieve can be directly a file name or any other text and the result is send by the server using the BODY. The very first request of the browser to a server is using the uri "/" the servers use to interpret as "/index.html" as default page.
(PC) (Smart phone)
<------------- "GET /" ---------------------|
------------- RESPONSE BODY ---------------->|
| <html> |
| <a href="nextPage.html">Next</a> |
| </html> |
<------------- "GET /nextPage.html" --------| User click on "Next"
------------- RESPONSE BODY ---------------->|
| .... etc
0- A first HTTP server
The samples explained in this article build both the server and client side of the application. In both cases the GUI (graphic user interface) and the logic are separated in two big groups or units called javaj and listix. Adding a third unit "data" which is common to both GUI and logic we have finally the general structure
variables to define GUI elements and its layout
variables to hold data of GUI elements and being accessible by listix
variables to define the logic containing text and commands
In this first very simple HTTP server we use pure html for the client, so the focus will be only on the server side.
The purpose is just serve a initial page containing a link to a second one which in turn has also a link to the first one. Although the simplicity it is actually a multi page application (MPA) because the browser is loading a new page each time. Next examples will be Single Page Application (SPA) which requires a little bit more programing in server and client sides.
The command needed to start the server is called MICOHTTP (or simply MICO) and has the syntax
MICO, START, serverNamne, portNumber
if we don't specify a portNumber one avaiable is chosen by the system, generally between ~ 50000 to 65000. Also in this case MICO will open automatically a browser pointing to the port so a first request to the server is immediately sent by the browser. This is very convenient for developing and testing phases but of course the last intention is to communicate from a browser of a device.
So if we start our server with the command
MICO, START, myFirstServer
we will have following communication scenario
(PC) (Web Browser in PC)
<------------- "GET /" ---------------------| Web browser opened by the MICO command
 | pointing to localhost:50130 (the server)
Notes: localhost is the name equivalent to the IP 127.0.0.1
ports numbers 50130 and 56380 are in this case chosen by the system and browser
On giving an address to the browser this sends immediately a first request to the server
To connect to the server from another device in the network (e.g. via WiFi)
we need to know the IP address of the server, localhost or 127.0.0.1 will not work!
The way to program the server to respond to a request is to put the response body in a variable named like the specific request (METHOD and path). In this case both pages will be static texts, which can be written as
Finally let's put a console as GUI for the server. Although real servers have actually no GUI at all this will help us for testing as well as being a way of stoping the server when we close the window.
Putting all together in a gastona script (firstServer.gast)
MICO, START, myFirstServer
now by typing in the command line
java -jar gastona.jar firstServer.gast
we have our first server runing.
In this example we are serving the pages directly from variables but the pages can be placed in physical files as well. Supposing that we name index.html the first page and nextPage.html the second one, the script itself will be reduced then to
MICO, START, myFirstServer
Now in the response to the first request we load the contents of index.html but nothing is needed for further requests referred to physical files that the server might find, since these will be served by MICO directly from the file. This means that if index.html has links to other pages, images etc that are to be found in the file system the above script would be enough to act as a server.
Sites based only on files (html, css, js, gif etc) that not change are referred as static web pages, now we know how to achieve it using MICO HTTTP server.
The next three examples explained will be SPA or Single Page Applications.
Loading JAST script for the Client part
First the library has to be loaded in the browser, then we initialize jGastona with a special script, let's call it JAST, which will define the client application in each case.
As we will see the JAST script, used for the client, and the gast script used for the server are very similar in form but actually they are completly different implementations where one runs as a desktop application and the other just in the browser.
Following code for the server do all the needed loads in one step just in the first response.
where META-GASTONA/js/jGastonaEva-min.js is contained in gastona.jar so it will be loaded from there and MAIN_JAST is the name of a variable where we have to put the JAST script for a particular application.
Note that the html body that we provide contains no html element at all except the script tag. The magic happens in jGastona that based on the JAST we provide will generate all the elements on the fly directly in the browser.
Now we have to programm on each case the JAST script and the server logic by defining the responses to the rest of http requests that can be sent.
1- Screenshot from server
This utility has the purpose of getting a screenshot of the PC where the server is running and send it back to the client. So, for example, we can check the progress of some task running on the PC from the kitchen using a smartphone.
Client side (JAST)
For the GUI of the client we just need a button and an image element. We give them a name for further reference, the first caracter or characters indicates the type of element, in this case "b" for button and "m" for image. We dispose them in a special grid layout called EVALAYOUT as follows
<layout of main>
EVA, 12, 12, 8, 8
X , mImagen
when the user press the button, the server has to do a new screenshot and send the new image to be updated in mImagen. Also from the client point of view this can be described simply as : on pressing the button, update the content of mImagen.
Given that the button event is captured in the variable "-- bScreenshot", following code will do the job.
so when the button "bScreenshot" is pressed we call the jGastona function AJAXgetIdContent which asks the server for updating the image "mImagen" or better said the source of the image.
Putting all together in the variable MAIN_JAST
The server has to respond to the call done by the client through the function AJAXgetIdContent which is nothing but the request
POST /getIdContent?id=mImagen HTTP/1.1
A first (but naiv) try to do so would be
SCREEN, myScreenshot.png, png
where the first line is the command SCREEN that makes a screenshot over the file myScreenshot.png of png type. Note that it is executed but it does not generate actually any text to be added to the body, while the second line is a text (only one element in the row) and will form the response body of the getIdContent request.
This will work, but only the first time! Why ?
After each screenshot the browser should perform the request "GET /myScreenshot.png" so the server can deliver the new content of the file. But this only happens the first time, next requests are taken from the browser cache instead, and the server does not receive the GET anymore.
To avoid this caching mechanism we will rotate the name and remove the old screenshot to keep the directory clean. It is not the only solution but it will work with any browser and also ilustrates how to call system functions.
NUM=, NN, NN + 1
SCREEN, @<SHOT_NN_PNG>, png
the first command is a CALL for windows to delete the old screenshot file, for linux or MacOS we should write
Note that for windows we use "@<:path ..." that changes the "/" to the native separator (in windows "\") in order for the "del" command to work. We haven't write directly "\" in the variable SHOT_NN_PNG because for the browser we need the separator "/" even if the server runs in windows.
Listix distinguish commands from text because commands and its parameters and options are always expressed in more than one column separated by commas, therefore a row containing only one value or column is a text.
Note also the use of "//" this time not at the begining of the row but within a command. The sequence "//" has the special meaning of "take the value from here to the end of the line". It is usually needed when the value contains commas or other special characters like < or quotes but in general it can be always used for the value of the last column of a row.
micohttp, start, monoMico
NUM=, NN, NN + 1
SCREEN, @<SHOT_NN_PNG>, png
2- A file server with upload ability
In this example we allow downloading files located in a specific directory on the server and also upload one file at a time from a device. The directory where the files to download are placed can be configured in the script while the upload is done always in a fixed directory named "filesUpload" that has to be created or ensured on starting the script.
For the files to be downloaded we just have to generate am html table with links to the final files, then when clicking on them in the browser they will be requested and downloaded automatically.
For the upload feature we need two buttons one to select the file to upload and one to start the download.
Uploading will happen automatically in the server side, we still will code something in the client side for a basic feedback of the upload progress.
Client side (JAST)
In this GUI we also use a single list of component so our grid will be
, lAvailable files
X , dFileList
, lUpload a file
where the widget type used are
d div (to place the html table later)
u special button for selecting file to upload
b normal button
For the logic, we request initially the list of the files to download with
For the button to start the upload of the file, we use jGastona functions to make some checks, start the upload and once started we change the text of the button to "uploading ..." to provide some feedback and on receiving the server response restore the initial button label.
<-- ajaxResponse uploadFile>
First the server will receive the post
POST /getIdContent?id=dFileList HTTP/1.1
in order to serve the table with the links we use the command LOOP (LOOP, FILES) in all loop commands the default option is BODY (body of the loop) an it can be omited as we will do in further examples
LOOP, FILES, @<FIDIR>
, RECURSIVE, 0
, BODY ,
, BODY ,
, BODY ,
Note within the loop the use of the variable "FIDIR" which is declared before but more interestently the use of the variable "fileName" which is specific for this loop and change its value with each iteration.
Regarding the upload, if a file has been uploaded succesfuly the server set the variable _uploadedFile0 with the name of the physical file on the server side. So we can check it and return the appropiate response (ok or nok)
CHECK, VAR, _uploadedFile0, nok
CHECK, FILE, @<_uploadedFile0>, nok
although for the sake of simplicity we actually don't use this information in the client side
3- A simple bulletin stored in the server
The purpose of this utility is to have a bulletin of events stored in the server with date, title and description that the client can handle remotely.
For this script we will learn more advanced features from gastona and jGastona, namely
- Dynamic GUI layout switching
- Passing structured data (a form)
- Using SQL databases
In this example we have separated the JAST in its own file Boletin.JAST
Let's take a look at the JAST code combined with the result in the browser
In the javaj unit, using very few lines the GUI consisting of two different layouts is created, html elements and its positions, all except the widget dBoletin which table content is filled by the server.
jGastona will load initially the layout main. On pressing the "new Event" button the jGastona method mask("main", "layInput") is called and it makes "layInput" to be shown instead of "main". The contrary occurs ("main" layout is restored) when pressing the button "bCancel". All this happens only in the browser, no request or response from the server is needed to switch layouts.
Also important to note is that on creating html components variables with their names are created in the data unit. So for example when entering date, title and description on a new event the variables eEventDate, eEventTitle and xEventDesc are filled with the input values and when calling AJAXSend ("insertEvent") are sent to the server.
The server is as well quite compact as shown with functionalities grouped
Note that now the loop is over a SQL select getting the first 50 records. The variables "date", "title", "evendesc" and "id", last used in DEL_COMMAND, are filled on each iteration automatically by the LOOP command.
Finally for those who don't have experience with databases and SQL
We create the table with
CREATE TABLE IF NOT EXISTS boletin (
id INTEGER PRIMARY KEY AUTOINCREMENT,
where we specify the long type to the "id" column in order to be filled automatically by the database engine on each insertion. The rest of columns does not require data type in sqlite. By no providing types to columns when possible we keep our schema flexible and efficient.
For insert a new event the SQL is
INSERT INTO boletin (date, title, eventdesc)
VALUES ('@<:encode eEventDate>',
here is very important the pre-process ":encode" on every field which escapes several characters among others the quote and double quotes (' and "). You can experiment yourself what happens if for example :encode is not used in eEventTitle and we try to save an entry with title ' (one quote). The SQL will be then malformed and the insert will fail. Actually the well known attack named SQL-Injection is based on this kind of wrong programmed SQLs.
Through three small scripts we have developed some non trivial and actually useful functionalities using http protocol.
The samples of this article will be maintained in the github repository HttpSweetHttp for serving as reference of http applications using gastona MICO http command (server) and jGastona js library (client) either together or separately.