Streaming Real G2 SDK
Volume Number: 15
Issue Number: 8
Column Tag: Tools of the Trade
Streaming Media with RealProducer G2 SDK
by Damon Lanphear
Streaming media has enlivened the online world with dynamic, media rich content that
has introduced novel ways to communicate information via the Internet. Media streams
are essentially packetized, compressed media files that are conveyed in a continuous
serial stream across a network infrastructure. The streaming media model is
juxtaposed with the traditional download model for the retrieval of multimedia
resources across networks. Instead of waiting for a media file to download prior to
playback on the local machine, local playback of streamed media files can begin after a
brief period of buffering. By improving the response time of access to media files,
streaming media has opened up new doors to wonderfully unique multimedia
presentations across the Internet.
Web publishers can now embed live streaming video and audio into their sites,
enhancing the user experience with sound and video content. Likewise, creative content
creators are continuously devising unique streaming multimedia solutions to deliver
anything from corporate training resources to artistic presentations. Indeed, the
advent of streaming media has brought the wealth of multimedia that Macintosh users
have enjoyed for years on their desktops into the limelight of the world wide web.
Seattle, WA based RealNetworks, Inc. paved the way for the ubiquity of streaming
media on the Internet. The most recent iteration of RealNetworks' streaming media
viewer solution, the RealPlayer G2, now enjoys an install base of over 60 million
users on an impressive array of operating systems. The RealPlayer G2 is one of three
components of the RealSystem G2. RealSystem G2's tripartite structure consists of
software components for media production, media delivery and media playback. These
components exist as software applications; the RealProducer, RealServer, and
RealPlayer, respectively.
RealNetworks has provided a means by which software developers can extend the
RealSystem G2 for custom applications. Through the RealMedia Architecture Core SDK,
and the RealProducer Core SDK software developers can, among other things, develop
and deploy custom data types for the RealSystem G2, develop a unique top-level client
to the RealSystem, or create a novel production environment for RealMedia. The SDKs
provided by RealNetworks for the RealSystem G2 provide a pathway for the realization
of software developer's creative visions for streaming media delivery. RealNetworks
makes the RealProducer Core SDK and RealMedia Architecture Core SDK available for
download from their website at: http://www.real.com/devzone/tools/index.html
This article will introduce the RealProducer Core SDK. The RealProducer core SDK
exposes cross platform, object oriented C++ interfaces to a set of dynamically linked
libraries, which are referred to collectively as the "core". These interfaces afford
applications programmers the ability to author custom solutions that take advantage of
the RealMedia codecs. Programmers can use the SDK to convert existing media files, or
a live media signal into RealMedia format. RealMedia files can then be streamed via
RTSP or HTTP using the RealServer G2. Accordingly, the RealProducer core SDK is
targeted at C/C++ programmers who wish to extend their content creation tools with
the ability to export RealMedia. In so doing, applications programmers provide the
users of their product with the ability to reach a terrifically wide audience of
streaming media viewers.
The RealProducer Encoding System
Before walking through a demonstration application built on the RealProducer Core
SDK, the structure and interfaces of the RealProducer Encoding system will be
presented. The function of the RealProducer core is to take RGB or YUV video data, raw
PCM audio data, an image map, or set of events as input and produce a single RealMedia
file as output. Additionally, the RealProducer Core provides API level support for
SureStream technology. By utilizing SureStream, content creators can fold multiple
streams, each targeted for an individual bandwidth, into a single RealMedia file. The
RealSystem G2 can then use SureStream encoded files to automatically deliver the
media stream that will be most efficient for a particular user's network
infrastructure.
A RealProducer client application consists of input source(s), an encoding session
manager, and an encoding engine. The encoding engine is provided by the RealProducer
core, and is responsible for the conversion of the input source to RealMedia format.
The application is left with the duties of managing the encoding session, and handing
data from the input source(s) to the encoding engine in the appropriate format.
Being an object-oriented API, the RealProducer Core SDK abstracts the functionality
of the RealProducer Core into accessible classes. The classes of the RealProducer Core
SDK adopt a nomenclature such that all classes relevant to the API are prefixed with
IRMA, or Interface Real Media Architecture. The RealProducer Core SDK represents
the encoding engine as the "Real Media Build Engine" and a set of "Input Pins",
IRMABuildEngine and IRMAInputPin respectively, that correspond to each of the data
types outlined above. Raw data is passed through the Input Pins to the encoding engine
in packages called "Media Samples". The RealMedia Build Engine is therefore used in
conjunction with the Input Pins to pass Media Samples through the codecs to create
RealMedia output.
The RealMedia Build Engine
Management of individual encoding sessions is conducted through the RealMedia Build
Engine, or IRMABuildEngine. The RealMedia Build Engine maintains session properties
that are common to all Input Pins. The engine is also responsible for notifying the
Input Pins when encoding has begun or if an encoding process must be cancelled. And
lastly, the RealMedia Build Engine handles the initialization of the SureStream rule,
which determines how a specific clip will be tailored to the bandwidth considerations
of a user defined target audience.
The IRMABuildEngine class exposes a number of methods that are used to access the
services of the build engine. Due to the ubiquity of the build engine's services
throughout the encoding process, the methods of IRMABuildEngine will not be
enumerated here. Rather, the methods in question will be encountered as the core
components of the encoding process are explained. The reader can refer to the
RealProducer Core SDK documentation for the list and description of the
IRMABuildEngine methods.
Only one RealMedia Build Engine is needed per application. For each encoding session,
an application will modify the settings of the RealMedia Build Engine and its associated
Input Pins as are appropriate for that particular session. Once the application is ready
to terminate execution, the RealMedia Build Engine can be released.
Typically, objects that are relevant to the RealMedia Build Engine can be created
through a method call of the form Get
the requested class, on the RealMedia Build Engine. Alternately, a call to
IRMABuildClassFactory::CreateInstance() can be used to generate instances of a class
required by the application. CreateInstance() takes as its parameters a class
identifier, and a pointer for the requested object. The exclusive purpose of generating
needed class instances in this way is to allow the programmer to have access to the
interfaces of the API, without maintaining a reference to the RealMedia Build Engine.
The reader can refer to the RealProducer G2 Core SDK documentation for a list of the
relevant class ids, which typically take the form CLSID_.
Before encoding begins the RealMedia Build Engine must be initialized with a number
of session specific settings. Such attributes of the output file as the filename, title,
author and copyright information are referred to as "Clip Properties". Clip
properties are set on the RealMedia Build Engine through the Clip Properties object.
To create a Clip Properties object you can call GetClipProperties() on a RealMedia
Build Engine object. GetClipProperties() will return a Clip Properties object
populated with either the current Clip Properties of the engine, or default properties
if clip properties have yet to be set on the engine. The Clip Properties object provides
the following methods:
IRMAClipProperties::
Get/SetDoOutputFile()
Get/set whether or not the RealMedia Build Engine should create an
output file.
Get/SetOutputFilename()
Get/set the output filename.
Get/SetTitle()
Get/set the title of the clip.
Get/SetAuthor()
Get/set the author of the clip.
Get/SetCopyright()
Get/set the copyright of the clip.
Get/SetComment()
Get/set the comment for the clip that will appear in a dump of the
file.
Get/SetSelectiveRecord()
Get/set whether the clip can be selectively recorded by the RealPlayer
Plus.
Get/SetMobilePlay()
Get/set whether the clip can be played on mobile players.
Get/SetPerfectPlay()
Get/set whether the clip will use PerfectPlay buffering. RealPlayer
versions 4.0 and above automatically use PerfectPlay
Similarly, basic target settings are set on the RealMedia Build Engine through the
Basic Target Settings object: IRMABasicTargetSettings. Basic target setting can be used
by the engine to specify which bandwidth streams are to be generated through the
encoding process. By targeting specific bandwidths, the encoded RealMedia file can be
tailored for the highest quality delivery given the restrictions of the underlying
infrastructure.
The IRMABuildEngine class exposes the SetDoMultiRateEncoding() method through
which an application can set whether or nor the encoder will generate a SureStream
file as output. This method must be called on an IRMABuildEngine and passed TRUE as a
parameter in order for multiple bandwidth streams to be generated within a single file
as specified in the object. By setting MultiRateEncoding to TRUE, the encoder is enabled
to produce SureStream files.
There do exist a default set of target bandwidths that are defined in reference to
common network connection methods. This set includes: 28.8K, and 56K modems,
single and dual ISDN, xDSL/Cable modems, and Corporate LANs. For each of the
connection methods in this set, there is an associated bitrate definition of the given
network connection. The associated bitrate represents an average bitrate for the
specified connection method. A 56K modem, for example, is defined as having a 34
Kbps average bit rate by default. Default averages are intended to account for such
uncontrollable variables as network congestion and electro-magnetic interference that
can affect the quality of a continuous media stream.
The application can change these definitions through the Target Audience Information
Manager: IRMATargetAudienceManager. The Target Audience Information Manager also
provides access to more thorough definitions that are associated with each of the target
audiences outlined above. These definitions include such specifics as which audio codecs
to use for encoding video clips -i.e. in order to most efficiently utilize available
bandwidth for the audio track-. It should be noted, however, that the defaults set for
the target audiences have been researched and proven to be rather effective in
producing quality output. As a result, it is not suggested that these attributes are
altered unless there is a specific need to do so. Detailed information on using the Target
Audience Information Manager is available in the RealProducer Core SDK.
The output quality of the encoded media itself can be set through the
IRMABasicTargetSettings object. By setting the audio or video output quality, the
application is telling the codecs to employ certain techniques, which are implemented
"under the hood", in encoding the media input. Such encoding techniques can, for
example, expand or contract the frequency response range in audio playback, or
provide improved motion integrity in video playback. There is a range of audio and
video output qualities that can be set, these include:
Audio Contents
ENC_AUDIO_CONTENT_VOICE Voice only
ENC_AUDIO_CONTENT_VOICE_BACKGROUND Voice with background
music
ENC_AUDIO_CONTENT_MUSIC Instrumental music
ENC_AUDIO_CONTENT_MUSIC_STEREO Instrumental music in stereo
Video Quality
ENC_VIDEO_QUALITY_NORMAL [TOKEN:21364]andard video quality
ENC_VIDEO_QUALITY_SMOOTH_MOTION Smoothest motion
ENC_VIDEO_QUALITY_SHARP_IMAGE Sharpest image
ENC_VIDEO_QUALITY_SLIDESHOW [TOKEN:21356]ide show
Aspects of the output file can be further honed by using interfaces exposed by the
IRMABasicTargetSettings object. These methods are as follows:
IRMABasicTargetSettings::
Add/RemoveTargetAudience()
Add/remove specific target audiences from the list of currently
selected ones..
RemoveAllTargetAudiences()
Remove all target audiences from the list.
GetTargetAudienceCount()
Get the number of currently selected target audiences.
GetNthTargetAudience()
Get the Nth target audience in the list.
Get/SetAudioContent()
Get/set the audio content type. See the subsection entitled "Audio
Quality" above for a list. (default: Voice Only).
Get/SetVideoQuality()
Get/set the desired video quality setting. See the subsection entitled
"Video Quality" above for a list (default: Standard).
Get/SetPlayerCompatibility()
Get/set the desired player version that you wish to support with the
output (default: G2 RealPlayer).
Get/SetEmphasizeAudio()
Get/set the emphasis for switching down when under duress conditions.