Network in server and client

This describes how networking is implemented in the game, and how the data is being sent.


This is sub-layout for documentation pages

Top

1 General Overview

To send data, serialization is heavily used.
Each serializable class contains some data to be sent, but it also usually contains another serializable class instance inside.


There are certain types of ,,data chunks" being sent between client and server. They can be seen in the tables below


Note: As the project s being developed, it is impossible to addess here all classes / exact names of classes
1.1 Data sent from server to client
Name Class name Identifier in network.json
Handshake data packet HandshakeDataPacket NetworkJson.handshakeResources.id
Map init data packet GameMapOnConnectPacket NetworkJson.game.gameMapInit
Game / system related data GamePacket NetworkJson.game.id
Player specific game packet PlayerSpecificGamePacket [NetworkJson.game.id: NetworkJSON.game.playerSpecificData]
Network dictionary - 'network-dict'

1.2 Data sent from client to server
Name Class name Identifier in network.json
Player input, inventory events, equipment events ClientToServerPacket NetworkJson.game.clientToSeverData

2 The Network Dictionary

In order to send data through socket, there must be some identifier.

      socket.emit(identifier, {data});
      

The other side uses the identifier to listen for data:

      socket.on(identifier, function(data)
      {

      });
      

To avoid having hard-coded long identifiers on both client/server side, the identifier is taken from network.json file. It can be (from code) referred to as to a word. For example:


      socket.emit(NetworkJson.game.id, {classInstance.serialize()});

      socket.on(NetworkJson.game.id, function(data)
      {

      });

      //NetworkJson.game.id would result in being some number in fact.
  

Example of network.json file:


    "game": {
      "__comment": "Represents values for messages being sent during a game",
      "id": 0,
      "game_init": 0
    }
    

This is only an example, real network.json might be different



The Network Dictionary (= content of the network.json file) is sent using static hard-coded message identifier on both sides (client, server). When the client connects to the server, it loads all textures and resources. Then, it sends a message requesting the network dictionary. The server sends the network.json data on the request to the client.
After the client received the dictionary, further communication is possible.

3 Partial classes

Are classes, which are used by multiiple other classes (as the data can be sent in more packets). These classes are not depicted on all class diagrams, as it wold be alot of cut and paste, requiring rewriting multiple diagrams when then the system changes.
Instead, the partials along with the further hierarchy are listed below:


3.1 PlayerInformationSerializable

4 Types of sent game data

Some data with certain structure needs to be sent repetitively (during gameplay), some data needs to be sent only once (upon client connects), some data needs to be sent only when a game starts.
Because of that, these data structures are referred as Data Packets and are explained below.


4.1 Handshake data packet

Is sent to the client only after the client connects to the server.
The client sends a request for such data, and the server replies with the data chunk.
The data is created upon start of the server and NEVER changes.

It contains data taken from the database, such as all medals (id, resource directory/image name, display name ...), abilities (id, display name, resource directory/file name ...), monsters.
Therefore during gameplay, all that is necessary to be sent is the ID of the ability/medal/monster ... etc. The additional data will be already on the client side.


4.2 Map initialization data

This chunk contains data that is necessary to be transferred to client to inform him about the map. It is divided to 2 parts: static and dynamic. Each map segment has it's own pair of static and dynamic data.
only when game starts, and the data will be cached in client and (usually) never change.
This applies to map items (static ones)
alternatively, it can contain data such as basic info about players, their starting points and names...


4.2.1 Static

Static map data is data, which never changes, no matter the situation. This includes (but is not limited only to)

Client side caches any received static data, and then never asks server to send static data again (if client has that data cached)


4.2.2 Dynamic

Dynamic map data is data, which always changes (or can change). This includes (but is not limited only to)

That data is sent always to the client side, and client side never caches such a data. That means, that if player would transition from map \(A\) to map \(B\) 50 times, then 50 dynamic packets would be sent to this player.


The class composition structure is represented by the following class diagram.


4.3 General game data

This chunk contains data that is common for all players.
It is sent during gameplay every time when position/health .... changes.
In the project, it is represented by GamePacket class


example of data that is sent:

The class composition structure is represented by the following class diagram.
Please note, that diagram focuses on displaying the nested composition hierarchy, therefore most of variables, that are types of nonserializable class (such as data containers, arrays, numbers, strings) are not displayed as class members, (instead, are referred as ...variables...)

Considering the diagram above, serialize() is called upon GamePacket instance, and that calls serialize() on all its members.
Like this, the data is serialized through the tree, until leaf is encountered, or until the node has no data to serialize


4.4 Player specific data

The class composition structure is represented by the following class diagram.

This chunk contains data that only specific player needs to know, but not necessary for other players to receive.
alternatively, it is used to exclude some player: If some player shall not know about dropped item, it is not sent to him


example:


For example, other players do not need to know about other player's inventory status or that player picked up item to his inventory.
The data is being sent whenewer position / inventory status / equipped item status .... changes.

5 Optimalization


5.1 Reducing amount of sent data

Reducing amount of sent data even by one byte helps to increase performance greatly, considering that such data is sent every server update to all clients.

- Jan Glaser


5.1.1 Default variables

Let's imagine class containing 2 variables, damage (number), and isCritical (boolean), signifiing that the damage was / not critical.
We want to transfer the data of such a class to client, using serialization.

Now, we could send the number and the boolean every single time, but it can be done better.
Let's define, that if we do NOT send in the packet the isCritical value at all, we will (as packet receiver) assume, that damage was not critical (which is going to happen more frequently, than critical damage).
As result, there is higher probability to send less data (when damage was not critical).

Same approach could be applied to damage aswell, assuming, that \(0\) would be default value. That would then result in fact, that sometimes, we would sent empty packet (= empty list).
This approach can be used to most of data, and it heavily increases performance.

Note: Since I have no idea how exactly socket.io sends / optimizes sent data, I rather send number \(1\) instead of sending booleans, in case socket.io would send it as string...


5.2.1 Use De-Morgan's law

To check, if packet is empty, consider following code:

        public isEmpty(): boolean{
          return !this.containsDataA() &&
                 !this.containsDataB();
        }
    
Now, if the packet is empty, and even if it is not, \(2\) operations are done, if isEmpty is called.
That is because of the conjunction.
Using De Morgan's laws for propositional logic and boolean algebra, the code above can be rewritten as follows, not changing it's functionality:
        public isEmpty(): boolean{
          return !(this.containsDataA() ||
                   this.containsDataB());
        }
    
Here, if packet is empty, then \(2\) operations are done, because both terms in disjunction must be checked. However, if packet is not empty, then only \(1\) operation is done, this.containsDataA in best case


5.2.1.1 Conclusion

It can be said You save one operation, what is the deal?

However, let's take following to consideration:

Considering what was said above, reducing \(30\) operations to \(1\) in best case, we save \(29 * N\) operations every 20ms.
Ofcourse, the packet does not have to be added at all, if it is empty, but it is better to have some safety checks, and nice code, if possible, but this idea can be applied to many other situations!

6 How exact game data is sent


6.1 Sending the map data

This describes how is map data sent to client which connects to the map.


6.1.1 Decoration items

Decoration item is item, which is not usefull to server at all. The only point is to tell where to draw what texture. Server gathers and keeps array of useable items data to only be able to send it to clients.


For decoration items this process is simple, as is sent following:


6.1.2 Useable items

In order to send useable item data, a container (packet class) is filled with the data to be sent (id, position...etc.). When client side receives the packet, it creates it's own client version of useable item.
There can be also other useable item types, such as Door, Chest, and so on, but they all are sharing same base, which can be seen here:

The point is, that client needs only some fixed set of information to receive, regardless the useable item real type...


6.1.2.1 Why this suffies?

The point is, that client does not need to know anything about the useable item at all. Is it door, or chest box? It does not matter, since client side only needs to know following:

When the item will be used by client, all that is really happening is that ID of that object is sent to server, in some packet denoting that useable item is requirted to be used.
Exceptions:
If client really needs to know the real type of the item (bank / door) for drawing some map icons, then a container fill of door, (or full of bank useable items) is sent.


6.2 Game map static and dynamic data caching / sending

Can be found here

7 Class cache

The following information is no longer up to date, since the game changed.
Additional info:
In version dev - 1.5.6 and higher, the serialization of UseableItem, and others is no longer used. This means that client has its own Client side class versions, therefore no caching has to be resolved...

In order to send data to other side, some objects (that are inheriting from GameObject, for example), are caching information about their state change.
When packet is sent, this information is extracted and sent to clients.
Then, the cache is deleted.


7.1 Diagram

The situation is illustrated by the following diagram:


7.2 Disabling cache

Because majority of classes that are on server (GameObject, Character, Player, Monster) are shared sources between on client as well, damaging character (as result of received message from master server) would just build up cache in these objects, and as no data from cache is sent from client to server, it would just build up memory.
Because of that, classes posses function like disableNetworkCaching(), that will cause the object not to store any information for network packets.
It is designed to be called as soon as upon object creation (for example: create Player instance, and call the function afterward from outside), however, technically, this would work when called at any time (which is not recommended for the sake of clean code).

8 Position synchronization

Given timestamp \(t\), synchronization is a mechanism that makes sure that all clients will have exactly same (or close to) state as server in time \(t\).
Because of latency, it is literally impossible to get exactly same state in time \(t\), so that it why we are also satifsied with close to aswell.


8.1 Example of problems that emerge with incorrect synchronization

Here are cases, in which the implementation is not really correct. This is to let reader understand the consequences of, otherwise correctly, appearing solution.


8.2 Example 1

Suppose, that we are synchronizing character movement.
Suppose, that blue character wants to move to target point \(C\). Client send request to server, where server runs BFS, and finds the movement path. Then sends to client response, that such movement can be done.
Client receives the message, and runs BFS locally to find path, and then will move the character to target point.
If movement is stopped on server, then server informs client, and client stops movement locally aswell
Situation is show on following sequence diagram:


Note: Movement to target point can take even minutes here

Pros:
Cons:


8.2.1 Problem

Suppose, that client sends request to server to move.
Server will compute BFS path, and start moving player on server side.
Following image displays the status of board, where player position is denoted blue, target point as red.

Step 1:
Step 2:
Let's now consider that confirmation was sent to client, and client runs BFS and finds the path.
Also consider, that delivery of message will take \(2\) seconds, and (for some reason), BFS run on client side will take much longer than on server.
It is important to realize, that if user switches tab in web browser, the javascript execution stops! Like this, delay in execution of any time \(t\) can be created! ×
Step 3:


8.3 Example 2

Suppose, that server will be periodically sending to client his position (only if position changes).
In other words:

If position of player changed, send it to client in next nearest packet sending phase
This solution is good, and will provide good synchronization, as error in position will correct itself in next player movement.


8.3.1 Problem

The problem is, however, that tons of messages are being sent through network and considering alot of player, this is unfeasible.
It is important to stress, that socketIO does NOT use UDP protocol! That means, that each message is acknowledged, generating even more traffic.


8.4 A good solution?

There is question what is good solution, but this is the best I was able to come up with:
The good solution would be somwhere in middle of the previous two examples. Objectives:


8.4.1 The idea

The idea is to send position to client every time, when character moves to a tile.
If we consider X position \(x=0\), where on that position is centre of tile \(A\), and position \(x=1\), where on that position is centre of tile \(B\), and position would change as follows on server:
\(x=0,01\)
\(x=0,5\)
\(x=0,81\)
\(x=1\)
Then, packet would be sent only when \(x=1\), but not in previous cases.


8.4.2 Effect