Audio Playback Synchronization Over A Network

Published Date: 02 Nov 2017

Abstract – Concert is fully functional mobile application which can create and listen to already existing running concert and will synchronize themselves to have synchronized audio playback over a distributed network. Concert has a capability to discover already running Concerts and join them and listen to the song to get the audio playback synchronization accuracy in millisecond.

Keywords – distributed network, synchronization

Introduction

Concert lets users create a network of music players or speakers out of your phones. A group of people can play music available on their smartphones in sync with each other’s, thus creating a surround sound environment without the need of any heavy equipment a concert like environment or just for kids to have fun together of for serious folks to have sound playing in their own time, kitchen using just their phones. Application is divided in two sections. Application user interface and backend library which will provide the useful functions that can be called from user interface. In whole internship my task is to develop library which is completely written in C language so that it can be used for all the platforms iOS, Android and Ubuntu mobile OS. Current goal is to prepare application for iOS hence application user interface is written in Objective C. There were 3 main modules that required for listener to fully synchronize with the client.

Libsntp – Library for time synchronization

Filetransfer – Library for file transfer

Logger – Library for keeping statistics and data generated by application.

Below is the description of the library modules design, implementations and tests.

LIBSNTP

Libsntp provides all the data that client needed in order to synchronize their clocks with the server for example offset, time of server in mille seconds etc. There are many clock synchronization protocols (PTP, RBS) available but the most reliable protocol today is NTP which provides accuracy in milliseconds. Simpler version of NTP is SNTP (Simple network time protocol) a networking protocol for clock synchronization between computer systems over packet-switched, variable-latency data networks and embedded networks. Libsntp is nothing but implementation of SNTP. Client can use libsntp to synchronize their clocks with the running servers.

Clock synchronization algorithm

To synchronize its clock with a remote server, the NTP client must compute the round-trip delay time and the offset. The round-trip delay is computed as

, where is the time of the request packet transmission, is the time of the request packet reception, is the time of the response packet transmission and is the time of the response packet reception. is the time elapsed on the client side between the emission of the request packet and the reception of the response packet, while is the time the server waited before sending the answer. The offset is given by

The SNTP synchronization is correct when both the incoming and outgoing routes between the client and the server have symmetrical nominal delay. If the routes do not have a common nominal delay, the synchronization has a systematic bias of half the difference between the forward and backward travel times.

Design and Implementation

Libsntp is written in C language and designed in such a way that it can expose functions which can be used for creating client as well as server. As it is implemented in C every platform can reuse this implementation as a synchronization backend. Clients can give the IP address and port number of the server to synchronize with as input to the get the time offset.

Testing

Because of uncertain network traffic and delay in sending and receiving messages. We can’t really be sure about the round trip time it takes to send message. Only thing we can do is minimize the fluctuation in the delay. And decide the offset based on the average RTT delays.

FILE TRANSFER

File transfer is a generic term for the act of transmitting files over internet. There are numerous ways and protocols to transfer files over a network. Device or programs which provide a file transfer service are often called file servers. File transfer is the most important part of this application. Because of high transfer speed we decided to use UDP for transfer and implement reliability as per our requirement. We started with implementation using UDP sockets but it turns out that packet loss is huge in case of mobile wireless networks as a result we received file in corrupted format.

The reason we observed is the routers hold the packets to prevent the congestion in the network and sends the burst of packets to the device when congestion is low in the network but iOS devices can’t handle such huge burst and ended up losing the data. To incorporate reliability we implemented file transfer using TCP sockets and implementation turns out to be reliable and speed was far better than what needed. Future plan is to integrate secure way of transmission.

Design and Implementation

File transfer contains single module which will provide the functionalities for sending files to client as well as receiving files from server. Same module will be used by client and server. In Concert if user presses play button then user will become server. As there is limitation on the internet as well as C programming we can no send file directly to the client. Server will open the file and read the file in the form of fragments and send those fragments to the client. Client will discover the running servers using Discovery module (which will give the list of running servers in the network and return the IP address of the server selected by client) using that IP address and port number (we have a common port number for all servers) Clients will be able to connect with server to receive the file which is currently played server. Client will receive the file in the form of fragments and hook that fragment in the correct file offset to form the exact copy of the file sent by server.

Testing and usage

To test file transfer we need to create server and clients inside test and server will send file to client. And after receiving file client will use diff tool (generates the difference between two given files) to make sure that the received file is not corrupted.

File transfer Usage:

Server.c

#include <tcpfileshare.h>

int main(int argc, char** argv) {

char* filename= "test.mp3";

int port = 12345;

int s = sharefile(port, filename);

if (s) { Error(); }

return 0;

}

Client.c

#include <tcpfileshare.h>

int main(int argc, char** argv)

{

char* filename = "test.mp3"

char* ip = "123.456.789.012"

int port = 12345;

int s = recievefile(ip, port, filename);

if (s) { Error(); }

return 0;

}

LOGGER

Why need logger?

At first, logging application work may seem non-topical Perhaps, it’s just and atavism of those times when the result of a program’s work would be immediately printed? No, this is a very effective and often very essential method, allowing you to debug complex, parallel, or specific applications.

Debugging release-versions of an application

Debugging security mechanisms.

Logging is the only possible debugging method for an application, launched on the end user's computer. The accurate use of the log file will allow developers to get the full information necessary for diagnosis of problems.

Logging allows you to debug device drivers and programs for embedded systems.

Logging allows you to quickly detect errors after a batch launch of functional or load tests.

Logging enables remote debugging when interactive means are impossible or inaccessible. This is convenient for high-performance multi-user systems where the user puts his tasks into a queue and waits for them to be fulfilled. This approach is used nowadays in institutions and other organizations working with computing clusters.

Possibility to debug parallel applications. In such applications, errors often occur when creating a lot of threads or when there are problems with synchronization. Errors in parallel programs are rather difficult to correct. A good method of detecting such errors is periodical logging of systems which relate to the error, and examining of the log's data after a program crash.

My task was to implement logger which is flexible enough to provide functionalities to log data locally as well as on network. Programmer can enable and disable logs whenever they wanted. Features: Print in file, Print in console, Add/remove new log levels (Debug, trace, Fatal, Warning), rollback (upload the file to server when file size is more than given size), Zero performance overhead when the logging is disabled.

Design and Implementation

Every data user log should be inside one of these levels.

Trace, Debug, Info, Warn, Error, Fatal

Common macro LOG_LEVEL can be used in main file to enable and disable log levels depending on the requirements. Library will read LOG_LEVEL and using "and" binary operation extracts the log levels from LOG_LEVEL. There are multiple threads trying to print the log simultaneously which can lead to race conditions or deadlock situations. To mitigate this issue we used mutex which can prevent handling shared memory handled by multiple threads and allow only one thread at a time to print logs.

B. Testing and Usage.

Below are test cases that we implemented and tested using mock data.

Enable warn level disable all other levels

Enable fatal level disable all other levels

Enable debug level disable all other levels

Enable trace level disable all other levels

Disable console logs

Enable Logs in file and verify.

How to use the logger: Below is the usage that will only print log of two levels debug and fatal.

#include <liblogger.h>

#define LOG_LEVEL DEBUG|FATAL

Int main(int argc, char** argv)

{

LogDebug ("Debug Message");

LogInfo ("int val = %d \n", 5);

LogFatal ("Fatal: ptr %#x", 0xABCD);

return 0;

}

USING BACKEND WITH USER INTERFACE

This section describes how C backend library is used by user interface (Objective C) and how clients sync with the server to have synchronized audio play back.

Discovery Module:

Discovery module is used to discover the currently running concert players (server). When player start playing song it continuously broadcasts the packets containing the name and port number with some delay (~3seconds) in the background thread behind GUI thread. Those packets will be received by the clients (listeners). Client makes server list in the form of HashMap using name and port number as key value pair.

Application has two main modes. Player(server) and Listener(client) mode.

Player Mode:

In Player mode user can play songs and behave as a server simultaneously hence other concerts can join to player and start listening to the song currently played by server player.

To behave as server, player call sharefile function from filetransfer module with arguments port, filename. It will start streaming the file.

Listener Mode:

Listener mode uses discovery module to get the running servers list. Concert can select one of the players from the list and connects with it to listen currently playing song by players. When client will start listening to this server client will receive file using receivefile function from file transfer module with IP address, port and filename (to store) as arguments. To keep song synchronized, client will use libsntp to get the time offset between server and client and then client will send request to get current playing time of the song. Server will replay current time. After getting current playing time of the song, client will add/subtract time offset from current time of the song to get the actual time position where client should jump to have synchronized playback.

Conclusions

Concert tries to tackle music synchronization problem over a network with flexible user interface and easy configuration options to users. Having backend in C provides advantage over rewriting codes multiple times and flexible enough to run on almost all the platforms.

SCOPE AND FUTURE PLANS

Currently there is no encryption and decryption features in filetransfer. Plan is to incorporate security features to prevent data from middle men in the network.

Application backend has capability to become platform for developing applications. We are planning to extend platform if the application pleases the users in first release.

Acknowledgment

I would like to express my gratitude towards Prof. Prasenjit Majumder for giving me chance to work under his mentorship. I also like to thank Mr. Sudhanshu Agrawal (Grit Innovation) for being a guide at every stage.

REFRENCES

Synchronizing audio playback http://snarfed.org/synchronizing_mp3_playback