A Video Indexing System

Published Date: 02 Nov 2017

Abstract— Multimedia technology has been applied to many types of applications and the vast amount of multimedia data needs to be indexed. The usage of digital video data is very popular today. In particular, video browsing is a necessary activity in all kinds of fields. For effective and interactive exploration of large digital video archives there is a need to index the videos using their visual, audio and textual data. In this Paper, we focus on the visual and textual content of video for indexing. By this paper We have able to developed and designed a web-enabled Video indexing system with the help of video streaming based on a Windows 2003 streaming server platform. To create a video-indexing system we first create publishing points and playlists for each video clip that is to be indexed. Information stored in the corresponding wsx and asx files corresponds to the start and stop times of the clip on the source video file resident on the server. A PHP based back end on the web server accesses the video database which is implemented in MySQL. Through the web pages, users can submit keyword for search on the database. Back end scripts then generate a prioritized list (based on relevance search) for the users, who can then directly click the links of interest to view clips relevant to the search submitted.

INTRODUCTION

F or browsing, searching, and manipulating video documents, an index describing the video content is required. It forms the crux for applications like digital libraries storing multimedia data or filtering systems which automatically identify relevant video documents based on a user profile. To cater to these diverse applications, the indexes should be rich and as complete as possible. Until now, construction of an index is mostly carried out by document lists when people manually assign a limited number of keywords to the video content. The specialist nature of the work makes manual indexing of video documents an expensive and time consuming task. Therefore, automatic video indexing methods are necessary. Most solutions to video indexing use a unimodal approach, i.e. only the visual, auditory, or textual modality is used. Instead of using only one modality, multimodal video indexing strives to automatically classify (pieces of) a video

Documents based on multimodal analysis. In this we put forward a unifying framework for multimodal video indexing.

Streaming media adds engaging motion and sound to the Web experience, increasing site stickiness, interactivity, and retention. Streaming allows timely, dynamic content to be seen by a larger, even global audience, helping to cost-effectively disseminate information, to address new markets, and to bring your corporate culture closer to far-reaching constituencies. Streaming in its truest form, can help to protect video content from being "pirated" and misused. Streaming media is no longer merely a promise. Streaming is here today. This Primer won’t tell you everything about this rapidly emerging technology, but it will give you an overview of the opportunities and the pitfalls, the costs, and the basics. You’ll find out what makes streaming different, and how you can use compelling streaming media content to make your Web sites more dynamic. At the very least, you’ll learn enough to be an informed member of a work group planning and executing streaming implementations. But, even if you are a complete beginner, you’ll find out how easy it can be to edit, encode, and integrate streaming video into your own Web site. And, if you are already creating video productions, this Primer will introduce you to the state-of-the-art streaming media technologies you can use to extend

II. The Technology behind Video Streaming

Before creating digital video for streaming purposes, the needs to be certain that several factors exist. These include ensuring that we have (a) made the video in the proper format, (b) access to a streaming media server, and (c) adequate bandwidth to download and upload the video file. The hardware for creating video streaming materials requires much of the same equipment for creating traditional digital video no matter how the material is being used on the Web. These hardware and software components include a video camera (analog or digital), video editing program (e.g., Adobe Premiere, Final Cut Pro), video encoding software (e.g., Real Producer Plus, Discreet Cleaner), a computer with a lot of storage space to store the video files, and a high-speed network connection. Other equipment considerations include (a) obtaining a tripod for reducing the amount of movement that helps reduce the amount of work the encoder will have to do to compensate for the movement, (b) obtaining a directional microphone that is windproof to help reduce

background noise, and (c) ensuring adequate lighting. It is also important to make sure that the computer is powerful enough to meet the demands of digital video development. According to the Joint Information Systems Committee in its Click-and-Go Video Streaming guide, the computer should have the following specifications to adequately manipulate digital video for streaming purposes:

A high end PC (Intel or AMD Processor) with a processor speed at least 1.2Ghz+ for the PC.

At least 256MB RAM with a recommended of 512 MB.

A large and fast hard disk drive with a recommended of 80GBs+ and an RPM of 7200+.

A good quality graphics card with at least 32 MB of RAM.

Most streamed video comes in one of three popular formats: (a) Real Networks’ Real Media (b) Microsoft Windows Media (ASF or Advanced Streaming Format), and (c) Apple QuickTime. These three formats are proprietary in that media produced by one of the formats. All three have specific advantages and support the Real Time Streaming Protocol (RTSP). To stream files created in these three formats, the video files must be placed on a streaming media server that has the server-side software to stream the video. In addition, that server must have enough capacity and bandwidth to support the number of simultaneous video streams by multiple users. Other formats for streaming video include MPEG-2, and more recently, MPEG-4. MPEG-4 is a newer standard specifically developed to address Web and mobile delivery. MPEG-4 is backward-compatible with MPEG-2, a video compression.

A final consideration for creating streamed videos is the server. The RealPlayer is free and available for all major OS platforms. The RealNetworks’ Helix Servers are available on Windows, Linux, and Unix-based operating systems and support all major file formats, including RealMedia, Windows Media, QuickTime, MP3, MPEG- 4, and others. The Helix Basic Server is free for up to five simultaneous streams. The paid version of Helix Universal Server is a full-featured server that allows 100 simultaneous streams. Windows Media server and player are free, but they are only available for Windows OS only. Also, both the QuickTime server and player are free. The QuickTime player is available for most OS platforms, but the QuickTime server is supported natively in MAC OS. A server is needed to store the media file and software that can stream the data over the Web. The file must be uploaded to the server and linked to a web site so that students can watch the video. The user selects a file and the streaming process begins by sending small packets of information over the network to the client-side computer. Once enough data has been received, the player streams the video while the player acts like a buffer and collects the data at an uneven rate from the server. A generic streaming server would require the following specifications to properly house and run the videos: 1GHz+ processor, 1GB+ RAM, large hard disk storage capacity, and fast network connectivity. If the connection speed will not support the server, there is no point of having a fast server. Thus, choosing the fastest option available in terms of bandwidth is important because if the server does not have enough bandwidth to cope with the demands of the users, then students will receive poor video quality feedback such as jerky images, broken sound, and even loss of connection.

III. Advantages of Using Video Streaming

There are many advantages of using streaming video. Some of the advantages include (a) instant play, (b) distributing live events, (c) delivering long-forms of media, (d) multicasting to multiple viewers, and e) the easy creation of streamed files. Other advantages that streaming video files can offer educators are creating visually driven materials that are more appealing to learners, helping educators handle volatile or quickly outdated materials (e.g., CD-ROMs, tape) that can be stored into a searchable database, and create synchronized presentations by having audio accompany still images, graphics, or text . However, individual control of pacing is the main advantage of incorporating streamed videos into distance learning courses. With streamed videos, students can access the material asynchronously and independent of their location. Students are no longer bounded by the traditional classroom or the library to view visual materials provided by the instructor. With streamed videos, students can access the visual materials at home and at any time. Another control element is the choice over which material to observe on-demand. Finally, being in charge of when to start, pause, skip, and review the visual material is another way that students can contend with the material. In short, the primary advantage of streaming video is the ability for students to self-pace their learning.

IV. Limitation to Consider Video Streaming

However, there are drawbacks of streaming video that the instructor must consider. Inadequate bandwidth to retrieve streamed files is one limitation. Video streams can be bandwidth intensive. Depending upon the Internet traffic, users who have 56K modems or even cable/DSL connections may have difficulty in terms of retrieving and playing streamed video. Internet congestion can cause playback delays, and even living on a particular road or street can influence the reception of streamed video connection. In addition, competing technologies to control standards is another obstacle for video streaming. Finally, lack of training and technical support can prevent effective video streaming use. If support and training are not readily available, it is difficult to sustain streaming video in academic institutions because of limited access to technology and knowledgeable experts who can assist in maintaining and developing streamed video. Equipment requirements, technical support, and network infrastructures need to be firmly in place before any quality video streaming projects can occur. Therefore, it is vital that these resources are made available to take advantage of what streaming video can offer to education.

Cause to Study Video Indexing

Indexing video data is essential for providing content based access. Indexing has typically been viewed either from a manual annotation perspective or from an image sequence processing perspective. The indexing effort is directly proportional to the granularity of video access. As applications demand finer grain access to video, automation of the indexing process becomes essential. Given the current state of art in computer vision, pattern recognition and image processing reliable and efficient automation is possible for low level video indices like cuts and image motion properties.

Problem Analysis

An analysis of existing systems enabled us to determine the objectives of this project in more detail. Our primary goal was to develop a system to enable the collaborative indexing, browsing, annotation of video content between multiple groups at remote locations. In addition the system must support, High quality video – MPEG-2 files, Automatic shot detection, Hierarchical video segmentation , Flexibility – different domains, communities and metadata application profiles, International video metadata standards such as MPEG-7,Indexing of segments, shots, frames and regions within frames. Streaming Protocol Used to Access Video File

A data transfer protocol is a standardized format for transmitting data between two devices. The type of protocol used can determine such variables as the error checking method, the data compression method, and end-of-file acknowledgements. If all networks were constructed in the same manner and all networking software and equipment behaved similarly, only one protocol would be necessary to handle all of our data transmission needs. In reality, the Internet is comprised of millions of different networks running a wide array of hardware and software combinations. As a result, the ability to stream digital media content reliably to clients depends on a set of several well-engineered protocols. The protocols used to stream Windows Media-based content are:

Real Time Streaming Protocol (RTSP)

Microsoft Media Server (MMS) protocol

Hypertext Transfer Protocol (HTTP)

Windows Media Services manages the use of these protocols by using control protocol plug-ins. Windows Media Services includes the WMS MMS Control Protocol plug-in, the WMS RTSP Control Protocol plug-in, and the WMS HTTP Control Protocol plug-in. With the exception of the WMS HTTP Control Protocol plug-in, these plug-ins are enabled by default. The control protocol plug-in receives the incoming client request, determines what action is indicated by the request (for example, to start or stop streaming), translates the request into a command form, and then passes the command to the server. Control protocol plug-ins can also return notification information to clients if there is an error condition or a change of status. Basic networking protocols such as User Datagram Protocol (UDP) and Transmission Control Protocol (TCP) are used to manage more fundamental tasks such as network connectivity and packet error correction. The MMS and RTSP protocols are used in combination with both the UDP or TCP protocols.

V. Streaming Server System Overview

A streaming media system based on Windows Media Technologies typically consists of a computer running an encoder (such as Microsoft Windows Media Encoder), a server running Windows Media Services, and players. The encoder enables you to convert both live and prerecorded audio, video, and computer screen images to Windows Media Format. The server running Windows Media Services, called a Windows Media server, enables you to distribute the content over a network. Users receive the content you distribute by using a player, such as Windows Media Player.

Fig 2.9- Streaming media server overview

In a typical scenario, a user clicks a link on a Web page to request content. The Web server redirects the request to the Windows Media server and opens the player on the user's computer. At this point, the Web server no longer plays a role in the streaming media process and the Windows Media server establishes a direct connection with the player and begins streaming the content directly to the user.

The Windows Media server can receive content from several different sources. Prerecorded content can be stored locally on the server or retrieved from a networked file server. Live events can be captured using a digital recording device and processed through an encoder before being sent to the Windows Media server for broadcast. Windows Media Services can also rebroadcast content streamed from a publishing point on a remote Windows Media server.

VI. Time based video

With the help of the Windows 2003 Streaming server we create the time based video file means that we play and stop the stream file by the user choice,to create a time based video we need a playlist file in which we set the time of clipbegin and clipend the whole process is following.

Offline Indexing of Media File

For example, suppose we have a keyword as Hindusm, and the lecture begins 05 minutes in to the media file with the name Gurnank.wmv and last for 30 minutes.

Therefore the tuple for this would be Gurnank- GHM101-1-1-1-00:05:00.0( time is in hh:mm:ss.milisecond format)

Also suppose that we have another keyword as Surdas, the lecture for which it begins at 36 minutes in to GHM101-1-1-1.wmv(05 mins+30 mins of Gurnank+ 01 mins for gap).

Therefore the tuple for this would we Surdas-GHM102-1-1-1-00:36:00.0

2. Create on demand Publishing Point and assign the playlist

Assign a playlist file to it (call samplePlaylist.wsx). This step is done automatically by the add publishingpoint wizard if you choose the option to create the playlsit(and announcement).

to begin with just assign one media file(GHM101-1-1-1.wmv) to thet playlist.

3. If you open the playlist file, you would see that it has the following structure

<?wsx version="1.0"?>

<smil>

<media src="C:\WMPub\WMRoot\GHM101-1-1-1.wmv"/>

</smil>

The media element can have a number of attributes, one of which is the source destination src. There are other attribute also,one of which is some thing clipBegin. This attribute causes the media file to be timeshifted by clipBegin seconds when the playlist is played back.

The complete of the attribute names can be seen in Media Server too.

Click on a Publishing Point

On the right hand pane, You will see the 5 tabs(Moniter,Source etc.

Cick on source and you should see your playlist file appearing in the location and the elements of the playlist.

If you click on GHM101-1-1-1.wmv the right hand pan of this widow will display the playback details associated with this media file.

Changing or editing any of the attributes value here would cause the original playlist to get modified . For example , in this playlist details window , if we were assign a 10 second shift to clipbegin (assigned as 00:00:10.0), we will will see the following added in the playlist.

<?wsx version="1.0" encoding="utf-8"?>

<smil>

<media src="C:\WMPub\WMRoot\GHM101-1-1-1.wmv"clipBegin="00:00:10.0"/>

</smil>

4. On the other client side, the user still make a call to the publishing point thus the media file is played as a stream file.

VII. MySQL PHP Web Database: Create Database

Getting started with a database means first figuring out what you want to do. Our project we have create database of the following

Courseunit

CourseunitLecture

Courses

Serverinfo

Sessions

Videoindex

Table structure for table `courseUnit`

DROP TABLE IF EXISTS `courseUnit`;

CREATE TABLE `courseUnit` (

`courseid` varchar(6) NOT NULL default '',

`unitnumber` tinyint(1) NOT NULL default '0',

`unittitle` varchar(30) NOT NULL default '',

PRIMARY KEY (`courseid`,`unitnumber`)

) ENGINE=InnoDB DEFAULT CHARSET=latin1;

-- Dumping data for table `courseUnit`

/*!40000 ALTER TABLE `courseUnit` DISABLE KEYS */;

LOCK TABLES `courseUnit` WRITE;

INSERT INTO `courseUnit` VALUES ('GHM101',1,'Unit1');

UNLOCK TABLES;

/*!40000 ALTER TABLE `courseUnit` ENABLE KEYS */;

As we Create this database table for the Couse Unit we also create database for each data which we want to show on Web page.

VIII. Result

Media streaming is a technique for transferring audio and video in a way such that it can be processed as a steady and continuous stream. Streaming technologies are becoming increasingly important with the growth of the Internet because most users do not have fast enough access to download large multimedia files quickly. With streaming, the client browser or plug-in can start displaying data before the entire file has been transmitted. Streaming media enables real-time or on-demand access to audio, video, and multimedia content via the Internet or an intranet.

Although the developed system is completely functional, the following enhancements are necessary:

Keyword search has to be implemented to display results in order of relevance. For this purpose an appropriate relevance score has to be defined and search has to be organized accordingly.

The player and controls have to be embedded in the Web page itself.