1、MPEG-4,MPEG-4,MPEG-4, or ISO/IEC 14496 is an international standard describing coding of audio-video objects the 1st version of MPEG-4 became an international standard in 1999 and the 2nd version in 2000 (6 parts); since then many parts were added and some are under development today MPEG-4 included

2、 object-based audio-video coding for Internet streaming, television broadcasting, but also digital storage MPEG-4 included interactivity and VRML support for 3D rendering has profiles and levels like MPEG-2 has 27 parts,MPEG-4 parts,Part 1, Systems synchronizing and multiplexing audio and video Part

3、 2, Visual coding visual data Part 3, Audio coding audio data, enhancements to Advanced Audio Coding and new techniques Part 4, Conformance testing Part 5, Reference software Part 6, DMIF (Delivery Multimedia Integration Framework) Part 7, optimized reference software for coding audio-video objects

4、Part 8, carry MPEG-4 content on IP networks,MPEG-4 parts (2),Part 9, reference hardware implementation Part 10, Advanced Video Coding (AVC) Part 11, Scene description and application engine; BIFS (Binary Format for Scene) and XMT (Extensible MPEG-4 Textual format) Part 12, ISO base media file format

5、 Part 13, IPMP extensions Part 14, MP4 file format, version 2 Part 15, AVC (advanced Video Coding) file format Part 16, Animation Framework eXtension (AFX) Part 17, timed text subtitle format Part 18, font compression and streaming Part 19, synthesized texture stream,MPEG-4 parts (3),Part 20, Lightw

6、eight Application Scene Representation (LASeR) and Simple Aggregation Format (SAF) Part 21, MPEG-J Graphics Framework eXtension (GFX) Part 22, Open Font Format Part 23, Symbolic Music Representation Part 24, audio and systems interaction Part 25, 3D Graphics Compression Model Part 26, audio conforma

7、nce Part 27, 3D graphics conformance,Motivations for MPEG-4,Broad support for MM facilities are available 2D and 3D graphics, audio and video but Incompatible content formats 3D graphics formats as VRML are badly integrated to 2D formats as FLASH or HTML Broadcast formats (MHEG) are not well suited

8、for the Internet Some formats have a binary representation not all SMIL, HTML+, etc. solve only a part of the problems Both authoring and delivery are cumbersome Bad support for multiple formats,MPEG-4: Audio/Visual (A/V) Objects,Simple video coding (MPEG-1 and 2) A/V information is represented as a

9、 sequence of rectangular frames: Television paradigm Future: Web paradigm, Game paradigm ? Object-based video coding (MPEG-4) A/V information: set of related stream objects Individual objects are encoded as needed Temporal and spatial composition to complex scenes Integration of text, “natural” and

10、synthetic A/V A step towards semantic representation of A/V Communication + Computing + Film (TV),Main parts of MPEG-4,1. Systems Scene description, multiplexing, synchronization, buffer management, intellectual property and protection management 2. Visual Coded representation of natural and synthet

11、ic visual objects 3. Audio Coded representation of natural and synthetic audio objects 4. Conformance Testing Conformance conditions for bit streams and devices 5. Reference Software Normative and non-normative tools to validate the standard 6. Delivery Multimedia Integration Framework (DMIF) Generi

12、c session protocol for multimedia streaming,Main objectives rich data,Efficient representation for many data types Video from very low bit rates to very high quality 24 Kbs . several Mbps (HDTV) Music and speech data for a very wide bit rate range Very low bit rate speech (1.2 2 Kbps) . Music (6 64

13、Kbps) . Stereo broadcast quality (128 Kbps) Synthetic objects Generic dynamic 2D and 3D objects Specific 2D and 3D objects e.g. human faces and bodies Speech and music can be synthesized by the decoder Text Graphics,Main objectives robust + pervasive,Resilience to residual errors Provided by the enc

14、oding layer Even under difficult channel conditions e.g. mobile Platform independence Transport independence MPEG-2 Transport Stream for digital TV RTP for Internet applications DAB (Digital Audio Broadcast) . . . However, tight synchronization of media Intellectual property management + protection

15、For both A/V contents and algorithms,Main objectives - scalability,Scalability Enables partial decoding Audio - Scalable sound rendering quality Video - Progressive transmission of different quality levels - Spatial and temporal resolution Profiling Enables partial decoding Solutions for different s

16、ettings Applications may use a small portion of the standard “Specify minimum for maximum usability”,Main objectives - genericity,Independent representation of objects in a scene Independent access for their manipulation and re-use Composition of natural and synthetic A/V objects into one audiovisua

17、l scene Description of the objects and the events in a scene Capabilities for interaction and hyper linking Delivery media independent representation format Transparent communication between different delivery environments,Object-based architecture,MPEG-4 as a tool box,MPEG-4 is a tool box (no monol

18、ithic standard) Main issue is not a better compression No “killer” application (as DTV for MPEG-2) Many new, different applications are possible Enriched broadcasting, remote surveillance, games, mobile multimedia, virtual environments etc. Profiles Binary Interchange Format for Scenes (BIFS) Based

19、on VRML 2.0 for 3D objects “Programmable” scenes Efficient communication format,MPEG-4 Systems part,MPEG-4 scene, VRML-like model,Logical scene structure,MPEG-4 Terminal Components,Digital Terminal Architecture,BIFS tools scene features,3D, 2D scene graph (hierarchical structure) 3D, 2D objects (mes

20、hes, spheres, cones etc.) 3D and 2D Composition, mixing 2D and 3D Sound composition e.g. mixing, “new instruments”, special effects Scalability and scene control Terminal capabilities (TermCab) MPEG-J for terminal control Face and body animation XMT - Textual format; a bridge to the Web world,BIFS t

21、ools command protocol,Replace a scene with this new scene A replace command is an entry point like an I-frame The whole context is set to the new value Insert node in a grouping node Instead of replacing a whole scene, just adds a node Enables progressive downloads of a scene Delete node - deletion

22、of an element costs a few bytes Change a field value; e.g. color, position, switch on/off an object,BIFS tools animation protocol,The BIFS Command Protocol is a synchronized, but non streaming media Anim is for continuous animation of scenes Modification of any value in the scene Viewpoints, transfo

23、rms, colors, lights The animation stream only contains the animation values Differential coding extremely efficient,Elementary stream management,Object description Relations between streams and to the scene Auxiliary streams: IPMP Intellectual Property Management and Protection OCI Object Content In

24、formation Synchronization + packetization Time stamps, access unit identification, System Decoder Model File format - a way to exchange MPEG-4 presentations,An example MPEG-4 scene,Object-based compression and delivery,Linking streams into the scene (1),Linking streams into the scene (2),Linking str

25、eams into the scene (3),Linking streams into the scene (4),Linking streams into the scene (5),Linking streams into the scene (6),An object descriptor contains ES descriptors pointing to: Scalable coded content streams Alternate quality content streams Object content information IPMP information ES d

26、escriptors have subdescriptors to: Decoder configuration (stream type, header) Sync layer configuration (for flexible SL syntax) Quality of service information (for heterogeneous nets) Future / private extensions,terminal may select suitable streams,Describing scalable content,Describing alternate c

27、ontent versions,Decoder configuration info in older standards,cfg = configuration information (“stream headers”),Decoder configuration information in MPEG-4,the OD (ESD) must be retrieved first for broadcast ODs must be repeated periodically,The Initial Object Descriptor,Derived from the generic obj

28、ect descriptor Contains additional elements to signal profile and level (P if (predefined=0) bit(1) useAccessUnitStartFlag; bit(1) useAccessUnitEndFlag; bit(1) useRandomAccessPointFlag; bit(1) usePaddingFlag; bit(1) useTimeStampsFlag; uint(32) timeStampResolution; uint(32) OCRResolution; uint(6) tim

29、eStampLength; uint(6) OCRLength; if (!useTimeStamps) .,SDL- Syntax Description Language,Wrapping SL packets in a suitable layer,MPEG-4 Delivery Framework (DMIF),The MPEG-4 Layers and DMIF,DMIF hides the delivery technology Adopts QoS metrics Compression Layer Media aware Delivery unaware Sync Layer