Digital cinema reels from motion JPEG2000 advances

The format is uniquely positioned to build the studio systems and consumer electronics necessary to migrate digital cinema into the home. Essentially, JPEG2000 helps overcome the interoperability problems, lack of standardization, content protection issues, large storage requirements and necessity for high bandwidth networks that so far have undermined the proliferation of high-resolution digital cinema and home theatre.

It can provide compression ratios up to 40:1 providing a 100 percent improvement in compression efficiency over JPEG.

At the heart of JPEG2000 is a new wavelet-based compression methodology that imparts a number of benefits over the discrete cosine transform (DCT) compression methods used in JPEG. The wavelet encodes an image in a continuous stream. This circumvents artifacts that result when DCTs divide an image into discrete compression blocks. At extreme compression levels, any introduced wavelet artifacts take the form of blurring high contrast lines, merely making the image look softer to the viewer — a significant advantage over existing JPEG and MPEG formats.

One interesting innovation in JPEG2000 is the scalable, multi-resolution compression. Within an image file, information is stored in a series of bands where each band contains a representation of the entire image. The bands scale from coarse resolution and textures to fine details. Because the most important details are stored at the front of the file, the image can be accessed in a smooth, continuous fashion. Pictures appear first as an image with coarse resolution and then with finer resolution as details are progressively filled in.

Usually, with only about 10% of the image data, a user can already tell what the image will be and decide whether or not to wait for higher resolution. Basically, JPEG2000 provides a more efficient way of displaying and downloading images that eliminates frustrating waits. This is especially important for Web-based applications.

In addition, JPEG2000 offers Region of Interest (ROI) coding. The user can define regions within an image which need to be coded with a higher quality than the remaining parts of the image. These ROIs are placed early in the codestream so that they will be decoded or refined first in the decoding process.

JPEG2000 supports both lossy and lossless compression in the same file. The 9/7 wavelet and the quantizer are used for lossy compression and the 5/3 wavelet with no quantization is used for lossless, with the same entropy encoding of either set of data. For lossless compression, the wavelet can usually cut file sizes in half.

This is important for digital cinema and home theatre. With JPEG2000, the highest quality lossless compression can be used to archive 35mm reels and new direct-dig ital content, while a lossy level of compression can be used for transmission and display in the home. In situations where a movie is encoded for distribution to different devices, a single file can be created where the content can be extracted without the requirement for transcoding and with no data redundancy.

JPEG2000 allows huge images (up to 232 rows and columns, effectively no limit on the number of megapixels it can support), a variety of image bit depths, and up to 256 channels of information. This means that JPEG 2000 is capable of describing complete alternate color models within a single file format. Additionally the incorporation of meta-data, such as graphic ownership and copyright, in the file format means that intellectual property rights information can be maintained within the file format.

Network scalability

Unlike JPEG, JPEG2000 has a defined file format especially for motion applications (known as Part 3). This flexible file format is base d on the MP4/QuickTime format and allows for easy synchronization of the compressed image data with audio and other meta-data.

In the imaging marketplace, it is expected that motion JPEG2000 and MPEG will coexist. While MPEG is more compression efficient than motion JPEG2000, the latter is scaleable and better suited to networked and point-to-point environments. Also, it offers advantages with respect to random frame access. For digital editing, this ensures that movie stills can quickly and seamlessly be retargeted for applications such as graphic art.

In cinematography, creative movie content is also increasingly generated digitally using computer generated-animation packages. To incorporate these animations, the original film is digitally scanned, the animations are integrated into the content and, after digital manipulation everything is recorded back to film. Moving to an all-digital frame based file format will lead to significant time and cost benefits.

JPEG2000 error resili ence is achieved primarily through a packetized codestream. The packets of data can be segmented within the codestream and synchronization markers can be inserted to improve error resilience. Both features prevent errors that may occur in one part of the image from propagating into other parts of the image during decoding. The codestream can also be reorganized so that the most important information, such as the header, is confined to a particular location in the codestream. This information can then be protected during transmission by an appropriate channel-coding scheme. Many home theatre applications envision distributing video wirelessly about the home. JPEG2000's error resilience facilitates this goal.

Unfortunately, JPEG2000 is approximately five times more complex than JPEG, and benefits from hardware acceleration. Preliminary studies have established that with current silicon technology, it is impossible for software-only solutions to deliver the required level of performance for most systems in this market space.

Therefore, to take full advantage of JPEG2000 in digital cinematography, hardware acceleration is essential. JPEG2000 algorithms can be run in software for lightweight applications such Web browsing and single-image encoding/decoding, but high-resolution full-motion JPEG2000 is a heavyweight streaming application.

Most systems featuring JPEG2000 will include general-purpose processors and DSPs. While these processors have reached impressive levels of performance, designers are finding that coding advanced DSP systems is increasingly complex and that meeting target performance can be elusive. By off-loading computationally intensive JPEG2000 algorithms from the system processor into a dedicated core, the system processor can take on other value-added functionality in the system such as audio processing and ease-of-use operations.

Amphion's engineers have taken the approach of directly mapping optimized signal processing algorithms into custom stream-based archi tectures on silicon. Performance is optimized further by taking advantage of the characteristics of whatever semiconductor device technology is being targeted.

For the JPEG2000 hardware architecture, efficient memory utilization is a dominant issue affecting performance. In programmable logic implementations, the availability of large amounts of structured memory blocks and embedded DSP features — such as the M4K memory blocks and DSP blocks in Altera Stratix devices — alleviate this problem.

For the wavelet engine in JPEG2000, 8 DSP blocks and 24 M4K blocks facilitate a solution that achieves throughput performance of up to 50 MSamples/s in a Stratix programmable device. This throughput enables the compression of 5 Mpixel still color images in less than 300 milliseconds. Robust JPEG2000 performance in professional cinema encoders can therefore be achieved using high-performance FPGAs such as Altera's Stratix devices.

Tier processing

Imag es are encoded via a two-stage operation. The initial stage (Tier-1) for each image can be accomplished using a CS6510 JPEG2000 encoder. The encoder interfaces directly with the user's system CPU — a Nios embedded processor for example-acting as a bus master with configurable source and destination memory addresses for DMA operation. For Tier-2 processing, control software must iterate through the image components and form the output Bitstream (JP2 file).

To compress an image, the source data, in any color space format, is read into the CS6510 core. The forward discrete wavelet transform analyses the input data and passes it on to the quantizer where the first phase of data compression is enabled. When sufficient data has been quantized it is read, one codeblock at a time, and is passed to one of a number of entropy encoders. For JPEG2000, the entropy encoding is active over a horizontal bit-plane of an image codeblock, and accordingly, parallelism is required to accelerate system throughput. The standard Amphion JPEG2000 solution contains three entropy encoders, each assigned to an individual codeblock and scheduled in a round-robin scheme.

The entropy coding efficiently compresses the image data and the encoded bitstream is output in parallel with distortion metrics for the compressed data. These distortion metrics are examined by the Tier-2 software running on the main system processor and are used to organize and truncate the compressed image data to facilitate rate control. The software then re-orders the bitstream into user defined order, such as SNR progressive, based upon the rate-distortion information.

To complete the encoding the software builds the file header and file marker information and places this in the code stream along with the selected entropy coded data. The output is a complete JPEG2000 bitstream, which is fully complaint with Part 1 of the JPEG2000 standard that specifies a regular level of compliance for JPEG2000 similar to baseline JPEG. The soon-to-be -released Part 2 of the standard specifies more flexibility and extensions to JPEG2000. One such extension allows the use of user-defined wavelet transforms to improve compression efficiency for particular image types. Programmable logic implementation of JPEG2000 could then be reconfigured in the field to include different wavelet engines, allowing the user to change compression efficiency as needed.

JPEG2000 alone will not solve all the challenges in archiving motion pictures that consume terabytes of data. Even with a 40:1 compression ratio the data can exceed 25 Gigabytes. Continued progress in disc storage technology, such as the use of blue lasers and holography, will be necessary to make access to digital cinema releases both economical and convenient for the masses. The move to a standardized digital format film production chain will minimize problems with film degradation, transportation and pirating.

JPEG2000 is a compute-intensive DSP sub-system that offers critical advantages ove r other compression schemes. Hardware-accelerated performance is the key to successful development of real-time JPEG2000 solutions for applications such as digital cinema and digital home theatre.