Feb 01 QTToolkit
Volume Number: 17
Issue Number: 2
Column Tag: QuickTime Toolkit
Honey, I Shrunk the Kids
By Tim Monroe
Using QuickTime's Standard Image Compression
Dialog Component
In a previous QuickTime Toolkit article ("Making Movies" in MacTech, June 2000),
when we built our very first QuickTime movie, we used a couple of Image Compression
Manager functions to compress each video frame so that it (and hence the entire
movie) took up less space on disk. The size reduction was significant: simply adding
100 uncompressed frames to the movie would have resulted in a movie file that was
about 12 megabytes in size. Using JPEG compression, we were able to reduce the final
movie file size to about 470 kilobytes.
In that article, however, we cut some corners by hard-coding the compression type
when we called GetMaxCompressionSize and CompressImage. It would have been nice to
provide the user with a choice of compression algorithms, and indeed perhaps even an
indication of what any particular compression algorithm would do to the penguin
images. Happily, QuickTime makes this very easy to do, by supplying the standard
image compression dialog component. We can use this component to perform two main
tasks. First, as the name suggests, we can have it display a dialog box in which the user
can adjust compression settings for a single image. Figure 1 shows the standard image
compression dialog box.
Figure 1. The standard image compression dialog box.
The standard image compression dialog box contains a pop-up menu that lists the
available image compressors. It also contains a pop-up menu that lists the available
pixel depths supported by the selected compressor. Finally, the dialog box contains a
slider control for adjusting the image quality. As the user varies the compressor,
pixel depth, or image quality, the standard image compression dialog component
adjusts the thumbnail picture to show what the image would look like if compressed
using the selected settings.
The second main task that the standard image compression dialog component can
perform is to compress the image. That is to say, it can not only retrieve the desired
compression settings from the user, but it can also do the actual compression for us
(thereby saving us from having to call GetMaxCompressionSize and CompressImage).
For this reason, the component is sometimes also called the standard compression
component. Figure 2 shows the result of using the standard image compression dialog
component to compress our penguin picture using the PNG image compressor at 16
levels of grayscale and the highest available quality.
Figure 2. The penguin picture compressed with PNG grayscale.
In this article, we'll see how to use the standard image compression dialog component
to elicit compression settings from the user and to compress images using the settings
selected by the user. We'll also see how to use the standard image compression dialog
component to compress a sequence of images (for example, the sequence of images that
make up our penguin movie). We'll begin by taking a more focused look at compression
itself. While the basic idea is straightforward, there are a handful of concepts we'll
need to understand before we can start using the standard image compression dialog
component. Then we'll spend the rest of this article investigating the
compression-related parts of this month's sample application, QTCompress. The Test
menu for QTCompress is shown in Figure 3; as you can see, it contains only one menu
item, which compresses the image or sequence of images in the frontmost window.
Figure 3. The Test menu of QTCompress.
Compression
Compression is the process of reducing the size of some discrete collection of data,
presumably without unduly compromising the integrity of that data. The basic goal, of
course, is to be able to store the data in less space and to use less bandwidth when
transferring the data over a network. Particularly for multimedia content like large
color images, movies, and sounds, uncompressed data (also known as raw data) simply
takes up too much space on disk or too much time to transfer over a network. It's
almost always better to store and transfer compressed data, which is then
decompressed during playback.
In QuickTime, compression and decompression are handled by components called codecs
(which is short for compressor/decompressor). The available codecs effectively
define the kinds of compressed data that QuickTime can handle. Apple has written a
large number of codecs itself and also licensed some other codecs from third-party
developers. Ideally, it would be nice if QuickTime supplied both a compressor and
decompressor for every kind of data that it can handle, but sadly that isn't the case. For
instance, QuickTime can decompress and play MP3 files, but it does not include a
component that can compress sound data into the MP3 format.
For the present, we'll be concerned primarily with compression and decompression of
images and sequences of images. In this case, there are two basic kinds of compression:
spatial compression and temporal compression. Spatial compression is a means of
compressing a single image by reducing redundant data in the image. For instance, our
penguin picture has large areas of pure white; a good spatial compressor would encode
the image so as to avoid having to store a 32-bit RGB value for every one of those
white pixels. Exactly how the encoding is accomplished varies from compressor to
compressor.
Temporal compression is a means of compressing a sequence of images by comparing
two adjacent frames and storing only the differences between the two frames. It turns
out that many common sorts of video change very little from frame to frame, so a
significant size reduction can be achieved by storing a full frame of the video and then
the subsequent differences to be applied to that frame in order to reconstruct the
original image sequence. In QuickTime, the full frame of video is called a key frame,
and the subsequent frames that contain only the differences from previous frames are
called difference frames or delta frames. (Other media technologies use other
nomenclature. Key frames are also called intraframes, and difference frames are also
called interframes. MPEG calls key frames I-frames and has two sorts of difference
frames, B-frames and P-frames.)
Figure 4 shows a key frame (on the left) and the immediately following difference
frame (on the right). The difference frame is pretty worthless when viewed by itself;
but when the decompressor, using its special algorithms, applies the difference frame
to the key frame, we get the actual image displayed in the movie, shown in Figure 5.
Figure 4. A key frame and the following difference frame.
Figure 5. The movie frame as reconstructed by the decompressor.
In this particular case, the space savings are significant. In the movie file, the key
frame occupies 4076 bytes, while the difference frame occupies only 3120 bytes. The
next difference frame occupies only 1184 bytes. The kinds of space savings you'll see
depend of course on the actual content of the movie. For instance, a music video with
lots of fast cuts and motion will not compress as well temporally as a movie of paint
slowly drying.
In theory, a temporally compressed movie could consist of a single key frame followed
by a large number of difference frames. But in practice, key frames are interspersed
throughout the movie at predetermined intervals. This is because, to be able to draw
any particular frame, the preceding key frame and all difference frames following that
key frame (up to the frame to be drawn) must be processed. It would be prohibitively
slow to jump to a random spot in a movie, or play a movie backwards, if it consisted of
a single key frame and a bunch of difference frames. The maximum number of frames
that can occur before a key frame is inserted is the key frame rate. A compressor may
insert key frames more often than the specified key frame rate, however (for
instance, at a scene change, where there is very little similarity between one frame
and the following frame).
Note that spatial and temporal compression are not competing forms of compression.
Indeed, most QuickTime movies employ both spatial and temporal compression, since
the key frames of a movie are typically spatially compressed images. Note also that the
use of temporal compression forces us to revise our understanding of the data stored in
a QuickTime movie file. Up to now, we've tended to think of the movie data as a sequence
of images. Now we see that it's more accurate to think of the movie data as a sequence of
images (key frames) and changes to those images (difference frames). Only at
playback time (that is, after the movie data is decompressed) do we get an actual
series of images.
Compressing Images
The sample applications that we've developed so far in this series of articles can open
both QuickTime movie files and image files and display them in windows on the screen.
When the user selects the "Compress..." menu item in the Test menu, QTCompress
executes this code:
case IDM_COMPRESS:
if (QTFrame_IsImageWindow(myWindow))
QTCmpr_CompressImage(myWindowObject);
else
QTCmpr_CompressSequence(myWindowObject);
myIsHandled = true;
break;
As you can see, if the frontmost window contains an image, then QTCompress calls the
function QTCmpr_CompressImage (which we'll consider in this section); otherwise, it
calls QTCmpr_CompressSequence (which we'll consider in the next section).
The QTCmpr_CompressImage function is built mainly around two routines provided by
the standard image compression dialog component, SCRequestImageSettings and
SCCompressImage. SCRequestImageSettings displays and manages the standard image
compression dialog box (Figure 1), and SCCompressImage performs the actual
compression of the image data into a new buffer. (As you've probably guessed, all
functions provided by the standard image compression dialog component begin with the
letters "SC".)
Getting the Image Pixel Map
The first thing we need to do when compressing an image is to draw it into an offscreen
graphics world. We'll use the pixel map associated with that offscreen graphics world
in two ways. First, we'll pass it to the SCSetTestImagePixMap function to set the
thumbnail image in the image compression dialog box. Then, later on, we'll pass it to
SCCompressImage as the source image to be compressed.
As we've seen in earlier articles, we can use graphics importers to open and draw
image files. In fact, we already have an instance of a graphics importer component
associated with the image file; it's the one we use to draw the image into the window on
the screen (namely, (**theWindowObject).fGraphicsImporter). But here we're going
to create a new graphics importer instance to draw the image into the offscreen
graphics world. This is because we'll want to use the existing graphics importer to
redraw the image in the on-screen window inside of the modal-dialog event filter
procedure QTCmpr_FilterProc (defined later). It might in fact be possible to cleverly
juggle the graphics importer's graphics world (using GraphicsImportSetGWorld), but
I never managed to get that strategy to work properly. So let's create a new graphics
importer instance for the image to be compressed:
myErr = GetGraphicsImporterForFile(
&(**theWindowObject).fFileFSSpec, &myImporter);
if (myErr != noErr)
goto bail;
myErr = GraphicsImportGetNaturalBounds(myImporter, &myRect);
if (myErr != noErr)
goto bail;
Now that we know the size of the image, we can use this code to create the requisite
offscreen graphics world:
myErr = QTNewGWorld(&myImageWorld, 0, &myRect, NULL, NULL,
kICMTempThenAppMemory);
if (myErr != noErr)
goto bail;
// get the pixmap of the GWorld; we'll lock the pixmap, just to be
safe
myPixMap = GetGWorldPixMap(myImageWorld);
if (!LockPixels(myPixMap))
goto bail;
Finally, we need to draw the image into myImageWorld.
GraphicsImportSetGWorld(myImporter, (CGrafPtr)myImageWorld,
NULL);
GraphicsImportDraw(myImporter);
At this point, the offscreen graphics world myImageWorld contains the image to be
compressed.
Setting a Test Image
Now we want to display the standard image compression dialog box, to get the user's
desired compression settings. To do this, we need to open an instance of the standard
image compression dialog component, like so:
myComponent = OpenDefaultComponent(StandardCompressionType,
StandardCompressionSubType);
Before we call the SCRequestImageSettings function to display the dialog box on the
screen, we need to set the thumbnail picture (called the test image) that is displayed
in the top-right part of the dialog box, by calling the SCSetTestImagePixMap function:
SCSetTestImagePixMap(myComponent, myPixMap, NULL,
scPreferScaling);
Here, the first two parameters are the instance of the standard image compression
dialog component and the pixel map that contains the image. The third parameter is a
pointer to a Rect structure that specifies the area of interest in the pixel map that is to
be used as the test image. Passing the value NULL means to use the entire pixel map as
the test image, suitably reduced into the 80 pixel by 80 pixel area in the dialog box.
The fourth parameter indicates how the image reduction is to occur; it can be any of
these constants:
enum {
scPreferCropping
= 1 << 0,
scPreferScaling
= 1 << 1,
scPreferScalingAndCropping =
scPreferScaling |
scPreferCropping,
scDontDetermineSettingsFromTestImage = 1 << 2
};
You can also pass the value 0 in the fourth parameter to request the component's
default method of displaying the test image, which is currently a combination of
scaling and cropping. Personally, I prefer just scaling the image to fit into the
available space, so I've passed the value scPreferScaling.
Installing Extended Procedures
We're almost ready to call SCRequestImageSettings to display the dialog box. We need to
do just one more thing to configure the dialog box, namely install one or more callback
procedures that extend the basic functionality of the standard image compression dialog
component. (These are therefore called extended procedures.) We install these
callback procedures by calling the SCSetInfo function with the scExtendedProcsType
selector:
SCSetInfo(theComponent, scExtendedProcsType, &gProcStruct);
Here, gProcStruct is an extended procedures structure, of type SCExtendedProcs,
which is defined like this:
struct SCExtendedProcs {
SCModalFilterUPP [TOKEN:26217]lterProc;
SCModalHookUPP hookProc;
long refcon;
customName;
};
The filterProc field specifies a modal-dialog event filter function, to handle events
that are not handled by the standard image compression dialog component itself. As
we've done in the past, we'll provide a filter function that looks for update events for
our application's windows and redraws those windows accordingly. Listing 1 shows our
custom filter function.
Listing 1: Filtering events in the image compression dialog box
QTCmpr_FilterProc
static PASCAL_RTN Boolean QTCmpr_FilterProc
(DialogPtr theDialog, EventRecord *theEvent,
short *theItemHit, long theRefCon)
{
#pragma unused(theItemHit, theRefCon)
Boolean myEventHandled = false;
WindowRef myEventWindow = NULL;
WindowRef myDialogWindow = NULL;
#if TARGET_API_MAC_CARBON
myDialogWindow = GetDialogWindow(theDialog);
#else
myDialogWindow = theDialog;
#endif
switch (theEvent->what) {
case updateEvt:
// update the specified window, if it's behind the
modal dialog box
myEventWindow = (WindowRef)theEvent->message;
if ((myEventWindow != NULL) &&
[TOKEN:10349]yEventWindow !=
myDialogWindow)) {
#if TARGET_OS_MAC
QTFrame_HandleEvent(theEvent);
#endif
myEventHandled = false;
}
break;
}
return(myEventHandled);
}
This is a fairly typical event filter function. It looks for update events that are not
destined for the dialog box and sends them to the framework's event-handling function.
Notice that this step isn't necessary on Windows; on that platform, redraw messages
are sent directly to the window procedure of the affected window.
We can also install a hook function, which is called whenever the user selects (or
"hits") an item in the dialog box. We can then intercept those hits and handle them in
any way we like. A typical way to use the hook function is in connection with a custom
button in the standard image compression dialog box. Figure 6 shows the dialog box
with a new button labeled "Defaults".
Figure 6. The standard image compression dialog box with a custom button.
We install this custom button by specifying a name for the button in the customName
field of the extended functions structure. We can do that with these two lines of code:
StringPtr myButtonTitle =
QTUtils_ConvertCToPascalString("Defaults");
BlockMove(myButtonTitle, gProcStruct.customName,
myButtonTitle[0] + 1);
Then we can handle user clicks on this custom button inside our hook function,
QTCmpr_ButtonProc, defined in Listing 2.
Listing 2: Intercepting events in a hook function
QTCmpr_ButtonProc
static PASCAL_RTN short QTCmpr_ButtonProc
(DialogPtr theDialog, short theItemHit,
void *theParams, long theRefCon)
{
#pragma unused(theDialog)
// in this sample code, we'll have the settings revert to their
default values
// when the user clicks on the custom button
if (theItemHit == scCustomItem)
SCDefaultPixMapSettings(theParams,
(PixMapHandle)theRefCon, false);
// always return the item passed in
return(theItemHit);
}
This hook function is extremely simple; it just looks for hits on the custom button
(signaled by the constant scCustomItem) and then calls the SCDefaultPixMapSettings
function to reset the dialog box to its default values. Notice that the reference constant
passed to our hook function (in the theRefCon parameter) is expected to be a handle to
the pixel map we created earlier. We install this reference constant by setting the
refcon field of the extended procedures structure. Listing 3 shows our definition of the
QTCmpr_InstallExtendedProcs function, which we use to set up our extended
procedures.
Listing 3: Installing the extended procedures
QTCmpr_InstallExtendedProcs
static void QTCmpr_InstallExtendedProcs
(ComponentInstance theComponent,
long theRefCon)
{
StringPtr myButtonTitle =
QTUtils_ConvertCToPascalString(kButtonTitle);
// the modal-dialog filter function can be used to handle any
events that
// the standard image compression dialog handler doesn't know
about, such
// as any update events for windows owned by the application
gProcStruct.filterProc =
NewSCModalFilterUPP(QTCmpr_FilterProc);
#if USE_CUSTOM_BUTTON
// the hook function can be used to handle clicks on the custom
button
gProcStruct.hookProc =
NewSCModalHookUPP(QTCmpr_ButtonProc);
// copy the string for our custom button into the extended procs
structure
BlockMove(myButtonTitle, gProcStruct.customName,
[TOKEN:28025]ButtonTitle[0] + 1);
#else
gProcStruct.hookProc = NULL;
gProcStruct.customName[0] = 0;
#endif
// in this example, we pass the pixel map handle as a refcon
gProcStruct.refcon = theRefCon;
// set the current extended procs
SCSetInfo(theComponent, scExtendedProcsType, &gProcStruct);
free(myButtonTitle);
}
You'll notice that we've used the compiler flag USE_CUSTOM_BUTTON to indicate
whether we want to install a custom button in the standard image compression dialog
box. Some image compressors want to install an Options button in that dialog box, and
our custom button would prevent them from doing so. (See Figure 7, later, for a dialog
box that contains an Options button.) For this reason, we usually won't install a
custom button. But you should at least know how to do so.
Finally we can call QTCmpr_InstallExtendedProcs to install our extended procedures
and then SCRequestImageSettings to display the dialog box.
if (gUseExtendedProcs)
QTCmpr_InstallExtendedProcs(myComponent, (long)myPixMap);
myErr = SCRequestImageSettings(myComponent);
Compressing the Image
If the user selects the Cancel button in the standard image compression dialog box, then
SCRequestImageSettings returns the value scUserCancelled. Otherwise, if
SCRequestImageSettings returns noErr, we want to go ahead and compress the image.
Thankfully, we can do this with a single call to the function SCCompressImage.
myErr = SCCompressImage(myComponent, myPixMap, NULL,
&myDesc, &myHandle);
SCCompressImage compresses the specified pixel map using the current settings of the
specified standard image compression dialog component. It allocates storage for the
compressed image and returns a handle to that storage in the fifth parameter
(myHandle). It also returns an image description in the fourth parameter (myDesc).
We can write the compressed data into a new file by calling the application function
QTCmpr_PromptUserForDiskFileAndSaveCompressed, passing in the compressed data
and the image description. (See the file QTCompress.c in this month's source code for
the definition of this function.) Listing 4 shows our complete definition of
QTCmpr_CompressImage.
Listing 4: Compressing an image
QTCmpr_CompressImage
void QTCmpr_CompressImage (WindowObject theWindowObject)
GraphicsImportComponent myImporter = NULL;
ComponentInstance myComponent = NULL;
myImageWorld = NULL;
PixMapHandle myPixMap =
NULL;
ImageDescriptionHandle [TOKEN:28025]Desc = NULL;
myHandle = NULL;
OSErr myErr
= noErr;
if (theWindowObject == NULL)
return;
myErr = GetGraphicsImporterForFile(
&(**theWindowObject).fFileFSSpec,
&myImporter);
if (myErr != noErr)
goto bail;
myErr = GraphicsImportGetNaturalBounds(myImporter,
&myRect);
if (myErr != noErr)
goto bail;
// create an offscreen graphics world and draw the image into it
myErr = QTNewGWorld(&myImageWorld, 0, &myRect, NULL, NULL,
kICMTempThenAppMemory);
if (myErr != noErr)
goto bail;
// get the pixmap of the GWorld; we'll lock the pixmap, just to be
safe
myPixMap = GetGWorldPixMap(myImageWorld);
if (!LockPixels(myPixMap))
goto bail;
// set the current port and draw the image
GraphicsImportSetGWorld(myImporter, (CGrafPtr)myImageWorld,
NULL);
GraphicsImportDraw(myImporter);
// open the standard compression dialog component
myComponent = OpenDefaultComponent(StandardCompressionType,
StandardCompressionSubType);
if (myComponent == NULL)
goto bail;
// set the picture to be displayed in the dialog box
SCSetTestImagePixMap(myComponent, myPixMap,
NULL, scPreferScaling);
// install the custom procs, if requested
if (gUseExtendedProcs)
QTCmpr_InstallExtendedProcs(myComponent, (long)myPixMap);
// request image compression settings from the user
myErr = SCRequestImageSettings(myComponent);
if (myErr == scUserCancelled)
goto bail;
// compress the image
myErr = SCCompressImage(myComponent, myPixMap, NULL,
&myDesc, &myHandle);
if (myErr != noErr)
goto bail;
// save the compressed image in a new file
QTCmpr_PromptUserForDiskFileAndSaveCompressed(myHandle,
myDesc);
bail:
if (gUseExtendedProcs)
QTCmpr_RemoveExtendedProcs();
if (myPixMap != NULL)
if (GetPixelsState(myPixMap) & pixelsLocked)
UnlockPixels(myPixMap);
if (myImporter != NULL)
CloseComponent(myImporter);
if (myComponent != NULL)
CloseComponent(myComponent);
if (myDesc != NULL)
DisposeHandle((Handle)myDesc);
if (myHandle != NULL)
DisposeHandle(myHandle);
if (myImageWorld != NULL)
DisposeGWorld(myImageWorld);
}
Compressing Image Sequences
Compressing a sequence of images is not fundamentally different from compressing a
single image. We'll need to display the standard image compression dialog box, as
before, to get the user's desired compression settings. Then, however, instead of
compressing a single image, we'll need to loop through all the images in the sequence
and compress each one individually. We'll obtain our sequence of images by extracting
the individual frames from an existing QuickTime movie, and then we'll write the
compressed images into a new QuickTime movie. (In effect, we'll be converting a
QuickTime movie from one compression scheme to another; this operation is often
called transcoding.) So we'll have the added overhead of creating a new QuickTime
movie, track, and media, and of adding samples to the media by calling
AddMediaSample. We've done this kind of thing numerous times before, so that part of
the code shouldn't slow us down too much.
When we want to compress a sequence of images, we need to call
SCRequestSequenceSettings instead of SCRequestSettings. The dialog box that it displays
is shown in Figure 7; as you can see, it contains an additional pane of controls for
specifying the number of frames per second, the key frame rate, and the maximum
data rate (the number of bytes per second that can be processed).
Figure 7. The standard image compression dialog box for an image sequence.
When configuring the dialog box and when responding to its dismissal, we'll need to add
code to handle this additional information. So let's get started.
Getting the Image Sequence
As mentioned above, we're going to obtain our sequence of images by reading the
individual frames from the video track of a QuickTime movie. (Let's call this the
source movie.) We can get the source movie by reading the fMovie field of the window
object record, and we can get the source movie's video track by calling the
GetMovieIndTrackType function:
mySrcMovie = (**theWindowObject).fMovie;
if (mySrcMovie == NULL)
goto bail;
mySrcTrack = GetMovieIndTrackType(mySrcMovie, 1,
VideoMediaType, movieTrackMediaType);
if (mySrcTrack == NULL)
goto bail;
To make things look a bit cleaner, we don't want the movie to be playing while we are
compressing its frames into a new movie, so we call SetMovieRate to stop the movie.
We also need to keep track of the current movie time, since we'll be changing it as we
move from frame to frame through the movie. Later, when we're done recompressing
the frames of the movie, we'll reset the movie time to this saved value.
SetMovieRate(mySrcMovie, (Fixed)0L);
myOrigMovieTime = GetMovieTime(mySrcMovie, NULL);
Finally, we need to know how many video frames are in the source movie, so that we
know (for instance) how many iterations our loop should make.
QTCmpr_CompressSequence includes this line of code for counting the frames of the
source movie:
myNumFrames = QTUtils_GetFrameCount(mySrcTrack);
There are several methods we could use to determine how many frames the source
movie contains. Probably the best method is just to step though the interesting times
in the movie using the GetTrackNextInterestingTime function, as shown in Listing 5.
Listing 5: Counting the frames in a movie
QTUtils_GetFrameCount
long QTUtils_GetFrameCount (Track theTrack)
{
long myCount = -1;
short myFlags;
TimeValue myTime = 0;
if (theTrack == NULL)
goto bail;
// we want to begin with the first frame (sample) in the track
myFlags = nextTimeMediaSample + nextTimeEdgeOK;
while (myTime >= 0) {
myCount++;
// look for the next frame in the track; when there are no
more frames,
// myTime is set to -1, so we'll exit the while loop
GetTrackNextInterestingTime(theTrack, myFlags, myTime,
fixed1, &myTime, NULL);
// after the first interesting time, don't include the time
we're currently at
myFlags = nextTimeStep;
}
bail:
return(myCount);
}
For more discussion of GetTrackNextInterestingTime, see "Word Is Out" in MacTech,
November 2000.
Configuring the Standard Image Compression Dialog Component
As before, we need to open an instance of the standard image compression dialog
component and configure the initial settings of the dialog box. Opening an instance of
the component uses the same code we used in the case of a single image:
myComponent = OpenDefaultComponent(StandardCompressionType,
StandardCompressionSubType);
To configure the settings in the dialog box, we first want to turn off the "Best Depth
menu option in the pixel depth pop-up menu. This is because we are going to draw the
movie frames into a 32-bit offscreen graphics world, regardless of the pixel depth of
the original source images. A better approach might be to determine the maximum bit
depth used in the source images (by looping through the video sample descriptions of
the video frames) and then create an offscreen graphics world of that depth. (This
refinement, of course, is left as an exercise for the reader.) We can disable the "Best
Depth" option using this code:
SCGetInfo(myComponent, scPreferenceFlagsType, &myFlags);
myFlags &= ~scShowBestDepth;
SCSetInfo(myComponent, scPreferenceFlagsType, &myFlags);
Next, we want to allow the user to leave the frame rate field blank (in which case the
compression component will preserve the original frame durations of the source
movie). To do this, we need to specify that 0 is an acceptable value in that field. We do
that by executing these lines of code:
SCGetInfo(myComponent, scPreferenceFlagsType, &myFlags);
myFlags |= scAllowZeroFrameRate;
SCSetInfo(myComponent, scPreferenceFlagsType, &myFlags);
If the user enters a number in the frame rate field, we'll use that number as the new
sample rate for the destination movie.
Setting the Test Image
Before we display the compression settings dialog box to the user, we want to set the
test image. In the present case, however, we have an entire sequence of images to
handle, not just a single image. Which of those images shall we select as the test
image? Let's select the movie poster image, on the assumption that that image is
representative of the content of the entire sequence of images (that is, of the source
movie itself). So we can call GetMoviePosterPict to get a PicHandle to the test image:
myPicture = GetMoviePosterPict(mySrcMovie);
Then we can get the size of the poster image and create an offscreen graphics world
large enough to hold that image:
GetMovieBox(mySrcMovie, &myRect);
myErr = NewGWorld(&myImageWorld, 32, &myRect, NULL, NULL,
0L);
And, as before, we'll lock the pixel map of that graphics world:
myPixMap = GetGWorldPixMap(myImageWorld);
if (!LockPixels(myPixMap))
goto bail;
Next we want to draw the poster image into the offscreen graphics world. Since we've
got a handle to a QuickDraw picture, we can use the DrawPicture function to draw the
picture. First, however, we need to make sure to set the current graphics world to our
new offscreen graphics world and to erase the destination graphics world.
GetGWorld(&mySavedPort, &mySavedDevice);
SetGWorld(myImageWorld, NULL);
EraseRect(&myRect);
DrawPicture(myPicture, &myRect);
KillPicture(myPicture);
SetGWorld(mySavedPort, mySavedDevice);
Finally we are ready to call SCSetTestImagePixMap to set the test image:
SCSetTestImagePixMap(myComponent, myPixMap, NULL,
scPreferScaling);
Displaying the Compression Settings Dialog Box
Once again, we have a couple of things still to do before we can display the standard
image compression dialog box. For one thing, we need to install the extended
procedures; here we can use exactly the same application function as in the
single-image case:
if (gUseExtendedProcs)
QTCmpr_InstallExtendedProcs(myComponent, (long)myPixMap);
Next, we want to set some default settings for the dialog box. The standard image
compression dialog component can examine the pixel map that we just created and
derive some sensible default settings based on the characteristics of that image. So let's
take advantage of that capability:
SCDefaultPixMapSettings(myComponent, myPixMap, true);
Also, we want to clear out whatever default frame rate was selected by the standard
image compression dialog component. As we discussed above, we'd like to use the frame
rate 0, indicating that the frame rate of the source movie should be used. (The user is
free to change this rate, but at least we want the default value to be 0.) We can first
retrieve and then reset the current temporal settings of the component:
myErr = SCGetInfo(myComponent, scTemporalSettingsType,
&myTimeSettings);
if (myErr != noErr)
goto bail;
myTimeSettings.frameRate = 0;
SCSetInfo(myComponent, scTemporalSettingsType,
&myTimeSettings);
SCGetInfo and SCSetInfo expect temporal settings to be stored in a structure of type
SCTemporalSettings, defined like this:
struct SCTemporalSettings {
CodecQ temporalQuality;
Fixed frameRate;
long keyFrameRate;
};
Finally, we're ready to call SCRequestSequenceSettings to display the standard image
compression dialog box:
myErr = SCRequestSequenceSettings(myComponent);
if (myErr == scUserCancelled)
goto bail;
Adjusting the Sample Count
If we've made it this far in the QTCmpr_CompressSequence function, we have
successfully displayed the standard image compression dialog box and the user has
selected his or her desired compression settings. In the single-image case, we could
finish up rather quickly, by immediately calling SCCompressImage and then saving the
compressed data into a new file. In the current case, however, we still have a good bit
of work left to do. We need to retrieve the temporal settings - which may indicate a
new frame rate for the destination movie - and configure the destination movie
accordingly. Then we need to step though the frames of the source movie and compress
each frame in the movie. We'll begin by first retrieving the temporal settings selected
by the user:
myErr = SCGetInfo(myComponent, scTemporalSettingsType,
&myTimeSettings);
If the user wants to change the frame rate of the movie (as indicated by a non-zero
value in the frameRate field of the temporal setting structure myTimeSettings), then
we need to calculate the number of frames in the destination movie and the duration of
the destination movie. We can do that like this:
if (myTimeSettings.frameRate != 0) {
long myDuration = GetMovieDuration(mySrcMovie);
long myTimeScale = GetMovieTimeScale(mySrcMovie);
float myFloat = (float)myDuration *
myTimeSettings.frameRate;
myNumFrames = myFloat / myTimeScale / 65536;
if (myNumFrames == 0)
myNumFrames = 1;
}
Creating the Target Movie
Suppose now that myFile is a file system specification for the destination movie file
(perhaps we called our framework function QTFrame_PutFile to elicit that file
specification from the user). At this point, we need to create the destination movie file
and movie, as shown in Listing 6.
Listing 6: Creating a new movie file and movie
QTCmpr_CompressSequence
myErr = CreateMovieFile(&myFile, sigMoviePlayer,
smSystemScript,
createMovieFileDeleteCurFile |
createMovieFileDontCreateResFile,
&myRefNum, &myDstMovie);
if (myErr != noErr)
goto bail;
// create a new video movie track with the same dimensions as the
entire source movie
myDstTrack = NewMovieTrack(myDstMovie,
(long)(myRect.right -
myRect.left) << 16,
(long)(myRect.bottom -
myRect.top) << 16,
kNoVolume);
if (myDstTrack == NULL)
goto bail;
// create a media for the new track with the same time scale as the
source movie;
// because the time scales are the same, we don't have to do any time
scale conversions
myDstMedia = NewTrackMedia(myDstTrack, VIDEO_TYPE,
GetMovieTimeScale(mySrcMovie), 0, 0);
if (myDstMedia == NULL)
goto bail;
We of course want to copy the movie user data and other settings from the source
movie to the destination movie:
CopyMovieSettings(mySrcMovie, myDstMovie);
Also, we want to set the movie matrix of the destination movie to the identity matrix
and clear out the movie clip region. This is because the process of recompressing the
movie transforms and composites all the video tracks into a single, untransformed
video track.
SetIdentityMatrix(&myMatrix);
SetMovieMatrix(myDstMovie, &myMatrix);
SetMovieClipRgn(myDstMovie, NULL);
Finally, since we're about to start adding compressed video samples to the destination
movie, we need to call BeginMediaEdits:
myErr = BeginMediaEdits(myDstMedia);
if (myErr != noErr)
goto bail;
Compressing the Image Sequence
We are now ready to start compressing the frames of the source movie into the
destination movie. We are going to step through the source movie, draw the current
frame into our offscreen graphics world, compress that data, and then add the
compressed data as a video sample to the destination movie. To start things off, we call
SCCompressSequenceBegin:
myErr = SCCompressSequenceBegin(myComponent, myPixMap, NULL,
&myImageDesc);
SCCompressSequenceBegin initiates a compression sequence, using the current
settings of the specified component instance and the characteristics of the specified
pixel map. The third parameter is a pointer to a Rect structure that indicates what
portion of the pixel map we're going to be compressing; the value NULL indicates that
we want to compress the entire pixel map. The fourth parameter is a pointer to an
image description. The standard image compression dialog component will allocate
storage for that description and fill in its fields when it compresses some data for us.
In turn, we shall use that image description when we call AddMediaSample.
Next we need to erase the offscreen graphics world that we're going to be drawing
movie frames into and set the source movie to draw its frames into that graphics
world:
SetGWorld(myImageWorld, NULL);
EraseRect(&myRect);
SetMovieGWorld(mySrcMovie, myImageWorld,
GetGWorldDevice(myImageWorld));
From now on, until we reset the movie graphics world, calling MoviesTask on the
source movie will cause the current movie frame to be drawn into myImageWorld.
So let's start looping through the frames of the source movie. We begin by setting a
variable that holds the current movie time to the beginning of the movie and by
retrieving the duration of the source movie:
myCurMovieTime = 0;
mySrcMovieDuration = GetMovieDuration(mySrcMovie);
The remainder of our code in this section will occur inside of a for loop that iterates
through all the frames of the source movie:
for (myFrameNum = 0; myFrameNum < myNumFrames; myFrameNum++) {
// get a frame, compress it, and add it to the new movie
}
The first thing we want to do inside this loop is get the next frame of the source movie.
If we are not resampling the movie (that is, changing the frame rate), we can use
GetMovieNextInterestingTime to step us forward in the movie; otherwise, we need to
step forward by the appropriate amount, based on the desired new frame rate. Listing
7 shows our code for this step.
Listing 7: Getting the next frame of the source movie
QTCmpr_CompressSequence
if (myTimeSettings.frameRate) {
myCurMovieTime = myFrameNum * mySrcMovieDuration /
(myNumFrames - 1);
myDuration = mySrcMovieDuration / myNumFrames;
} else {
OSType myMediaType = VIDEO_TYPE;
myFlags = nextTimeMediaSample;
// if this is the first frame, include the frame we are currently
on
if (myFrameNum == 0)
myFlags |= nextTimeEdgeOK;
// if we are maintaining the frame durations of the source movie,
// skip to the next interesting time and get the duration for that
frame
GetMovieNextInterestingTime(mySrcMovie, myFlags, 1,
&myMediaType, myCurMovieTime, 0, &myCurMovieTime,
&myDuration);
}
Then we need to set the current movie time of the source movie to the time just
calculated and draw the movie into the offscreen graphics world:
SetMovieTimeValue(mySrcMovie, myCurMovieTime);
MoviesTask(mySrcMovie, 0);
MoviesTask(mySrcMovie, 0);
MoviesTask(mySrcMovie, 0);
(Why three calls to MoviesTask here? Partly it's to make sure that the current movie
frame is completely drawn before continuing with our work, and partly it's just
because that's the way the sample code on which QTCmpr_CompressSequence is based
was written. Consider this your first taste of QuickTime black magic. No harm results
from a few extra calls here.) If the user has requested that the data rate of the
destination movie be constrained, then we need to tell the standard image compression
dialog component the duration of the current frame, in milliseconds. We can do that
using this code:
if (!SCGetInfo(myComponent, scDataRateSettingsType,
&myRateSettings)) {
myRateSettings.frameDuration = myDuration * 1000 /
GetMovieTimeScale(mySrcMovie);
SCSetInfo(myComponent, scDataRateSettingsType,
&myRateSettings);
}
Finally we can actually compress the pixel map data, by calling
SCCompressSequenceFrame:
myErr = SCCompressSequenceFrame(myComponent, myPixMap,
&myRect, &myCompressedData, &myDataSize, &mySyncFlag);
If SCCompressSequenceFrame completes successfully, then myCompressedData will
hold a handle to the compressed data and myDataSize will be the size of the compressed
data. In addition, the mySyncFlag parameter will hold a value that indicates whether
the compressed frame is a key frame (0) or a difference frame
(mediaSampleNotSync). We will pass this value directly to the AddMediaSample
function, like this:
myErr = AddMediaSample(myDstMedia, myCompressedData, 0,
myDataSize, myDuration,
(SampleDescriptionHandle)myImageDesc,
1,
mySyncFlag, NULL);
And so we've completely handled a frame of the source movie.
Finishing Up
Once we've exited the for loop that iterates through all the frames of the source movie,
we need to do a little cleaning up. First, we need to close the compression sequence that
we opened by calling SCCompressSequenceBegin:
SCCompressSequenceEnd(myComponent);
SCCompressSequenceEnd disposes of the image description and compressed data handles
allocated by SCCompressSequenceBegin.
Next we need to close our media-editing session by calling EndMediaEdits:
myErr = EndMediaEdits(myDstMedia);
Then we proceed as normal, inserting the edited media into the track and adding the
movie resource to the destination movie file.
InsertMediaIntoTrack(myDstTrack, 0, 0,
GetMediaDuration(myDstMedia), fixed1);
myErr = AddMovieResource(myDstMovie, myRefNum, NULL, NULL);
And at this point, we can safely close the destination movie file:
CloseMovieFile(myRefNum);
We also need to close down the instance of the standard image compression dialog
component that's been doing all this work for us:
if (myComponent != NULL)
CloseComponent(myComponent);
Finally, let's restore the source movie to its original graphics world and movie time,
and dispose of the offscreen graphics world:
if (mySrcMovie != NULL) {
// restore the source movie's original graphics port and device
SetMovieGWorld(mySrcMovie, mySavedPort, mySavedDevice);
// restore the source movie's original movie time
SetMovieTimeValue(mySrcMovie, myOrigMovieTime);
}
// restore the original graphics port and device
SetGWorld(mySavedPort, mySavedDevice);
// delete the GWorld we were drawing frames into
if (myImageWorld != NULL)
DisposeGWorld(myImageWorld);
We have now successfully compressed our image sequence. (Whew!) I won't bother to
give the entire listing of the QTCmpr_CompressSequence function, as it would run for
several pages. You can find the complete story in the source file QTCompress.c.
Asynchronous Compression
If you run the QTCompress sample application and have it recompress a sizable movie
file, you'll notice that, after you dismiss the compression settings dialog box and the
file-saving dialog box, no other events are processed until the entire recompression
sequence is completed. The mouse continues tracking, but you cannot access the
application's menus; worse yet, the application windows are not updated. This is all
perfectly understandable, as we've jumped into a for loop and just keep processing
movie frames until we are finished.
One way to address this problem would be to display a progress dialog box, which we
could update each time though the for loop. For good measure, we could also see if any
of our windows need updating and redraw them if necessary. This would cost us some
time, but it would significantly improve the user's experience.
Another way to address this problem would be to compress the movie frames
asynchronously. That is to say, we could launch a compression operation and then
process other application events while we are waiting for the operation's completion
procedure to be executed. That would allow us to redraw our windows and even grant
the operating system some time to handle its own periodic tasks.
QuickTime has supported asynchronous compression for many years, but until
recently we had to use the lower-level services of the Image Compression Manager
(namely, CompressSequenceFrame) to do so. As we've seen, there is no parameter to
the SCCompressSequenceFrame function that allows us to specify a completion routine,
which we'd need in order to be able to use the standard image compression dialog
component asynchronously. QuickTime 5, however, introduces the new function
SCCompressSequenceFrameAsync, which we can use for asynchronous compression. In
this section, we'll see how to modify QTCmpr_CompressSequence to support
asynchronous compression.
Setting Up for Asynchronous Compression
SCCompressSequenceFrameAsync is exactly like SCCompressSequenceFrame, except
that it takes one additional parameter, a pointer to an ICM completion procedure
record. The ICM completion procedure record, of type ICMCompletionProcRecord, is
declared like this:
struct ICMCompletionProcRecord {
ICMCompletionUPP [TOKEN:25455]mpletionProc;
long
completionRefCon;
};
The completionProc field is a universal procedure pointer to our completion routine,
and the completionRefCon field is an application-specific reference constant. When we
first enter QTCmpr_CompressSequence, we'll declare and initialize some additional
local variables, like this:
#if USE_ASYNC_COMPRESSION
ICMCompletionProcRecord [TOKEN:28025]ICMComplProcRec;
ICMCompletionProcRecordPtr myICMComplProcPtr = NULL;
myICMComplProcErr = noErr;
myICMComplProcRec.completionProc = NULL;
myICMComplProcRec.completionRefCon = 0L;
#endif
Then, just before we enter the for loop, we need to set some codec flags:
#if USE_ASYNC_COMPRESSION
myFlags = codecFlagUpdatePrevious +
codecFlagUpdatePreviousComp +
codecFlagLiveGrab;
SCSetInfo(myComponent, scCodecFlagsType, &myFlags);
#endif
The codecFlagUpdatePrevious and codecFlagUpdatePreviousComp flags are used to
optimize temporal compression. The codecFlagLiveGrab flag is required to get
SCCompressSequenceFrameAsync to work correctly. (More QuickTime black magic.)
Once we're inside the for loop, we'll fill in the ICM completion procedure record:
if (myICMComplProcPtr == NULL) {
myICMComplProcRec.completionProc =
NewICMCompletionProc(QTCmpr_CompletionProc);
myICMComplProcRec.completionRefCon =
(long)&myICMComplProcErr;
myICMComplProcPtr = &myICMComplProcRec;
}
For our reference constant, we're passing the address of a local variable of type OSErr.
The idea is that we can undertake other processing inside the for loop as long as the
value of that variable is set to a known value; the completion routine, when it is
triggered, will be responsible for changing that value, as we'll see shortly. Before we
initiate the asynchronous compression operation, we'll set myICMComplProcErr to a
default value:
myICMComplProcErr = kAsyncDefaultValue;
Performing Asynchronous Compression
As we've seen, we can begin an asynchronous compression operation by calling
SCCompressSequenceFrameAsync, like this:
myErr = SCCompressSequenceFrameAsync(myComponent, myPixMap,
&myRect, &myCompressedData, &myDataSize, &mySyncFlag,
myICMComplProcPtr);
The parameters here are exactly those that we earlier passed to
SCCompressSequenceFrame, with the additional final parameter myICMComplProcPtr.
As with all asynchronous functions, SCCompressSequenceFrameAsync returns
immediately. When the compressor is finished with the compression operation, our
completion routine will be executed. Listing 8 shows our application's completion
routine.
Listing 8: Signaling the end of a compression operation
QTCmpr_CompletionProc
static PASCAL_RTN void QTCmpr_CompletionProc
(OSErr theResult, short theFlags, long
theRefCon)
{
OSErr *myErrPtr = NULL;
if (theFlags & codecCompletionDest) {
myErrPtr = (OSErr *)theRefCon;
if (myErrPtr != NULL)
*myErrPtr = theResult;
}
}
When the codecCompletionDest flag is set in the theFlags parameter, we know that the
compression operation has finished successfully. At that point, we can copy the result
code passed to our completion routine into the location specified by theRefCon. This
effectively changes the value of the myICMComplProcErr variable in our for loop,
where we are executing this while loop until that value changes:
while (myICMComplProcErr == kAsyncDefaultValue) {
EventRecord myEvent;
WaitNextEvent(0, &myEvent, 60, NULL);
SCAsyncIdle(myComponent);
}
As you can see, we're repeatedly calling WaitNextEvent to yield time to other processes
and SCAsyncIdle to yield time to the compressor component. I'll leave it as another
exercise to add the code necessary to handle application events (for example, update
events).
Weighing the Benefits
While adding support for asynchronous compression is not too terribly complicated,
you might be wondering whether it's worth the trouble. In other words, are we going to
see appreciable performance gains, or at least significant user experience
improvements? Sadly, the answer is "no", at least at the moment. First of all, not all
compression components support asynchronous operation. (Indeed, at this point, there
is only one Apple-supplied codec capable of performing asynchronous compression,
the H.263 codec.) You can still invoke compression components using
SCCompressSequenceFrameAsync, but SCCompressSequenceFrameAsync will not
return to the caller until the compression is complete and the completion routine has
been executed. In other words, it will run synchronously. Second, and perhaps more
important, even codecs that are able to compress asynchronously will not show any
real improvement when running on a single-processor machine.
Conclusion
The standard image compression dialog component is a prime example of the kind of
QuickTime APIs that I like best: it provides a high-level toolbox for handling a wide
variety of typical image compression tasks, allowing us to compress both individual
images and sequences of images. Moreover, it gives us a standard user interface for
eliciting compression settings from the user (rather in the same way that the Standard
File Package or the Navigation Services provide a standard user interface for eliciting
files from the user).
Credits
Much of the code in the function QTCmpr_CompressSequence is based on code in the
earlier sample code package called "ConvertToMovie Jr." (which was itself based on an
even earlier sample called "ConvertToMovie").
______________________________
Tim Monroe works in the QuickTime Engineering team at Apple. You can contact him
at monroe@apple.com.