About Containers and Codecs

There are many file formats, codecs and variants in the video world. To the beginner, these names and properties can be daunting. In this post, I will try to simply some of the terminology to get you started. It’s not that hard, really.

Data:

All image files are made of pixels. Each pixel has values of RGB, YUV, etc assigned to them. The matrix of pixels (e.g. 1920 pixels x 1080 pixels) is what makes an image. To the computer, however, it’s just a matrix of numbers. This is data.

This data can be in the form of images, moving images, audio, subtitles, whatever; but they are all just numbers to the computer. A bunch of values is data. It doesn’t have to make sense. Most of us shoot a lot of nonsense anyway – it makes sense to us but others find it useless. Think of data as ‘stuff or items’ filled in shopping carts that people use in large supermarkets.

File:

A collection of data is a file. Think of an office binder or folder with whatever you want to put in them. If you want to continue the analogy I’ve given in data, the shopping cart, is the file.

There are many well-known ‘types of files’ out there – called file formats in general. To distinguish between file formats, the standard procedure is to have the name of the file followed by a full-stop and then an extension of letters (usually three but this can vary). E.g., run.exe, photo.jpeg, mymovie.avi, etc.

Meta Data:

Metadata has many meanings, and as a term is used by different people with different intent and meaning. However, to the layman, all metadata means is: Data about the data. E.g., the JPEG images that come out of a digital camera have the lens, aperture, shutter, ISo, date, etc embedded in the file. This data is not really part of the image, but ‘extra’ data, or metadata.

Actually one could argue that metadata is also another kind of data. Of course, it is. But in the world of audio and video, it is the ‘extra’ data that gives certain information about the data in the files that are not in the files themselves. No need to worry about the definition of this term.

Codec:

As most of you know, video comes in compressed and uncompressed. You will have guessed that we need to compress to save hard disk space. Compression also helps while streaming video through the internet or television because bandwidth is limited.

Codec is COmpression-DECompression/COding-DECoding – a system in which a file (bunch of data) is compressed/coded in a certain way (using a certain technology or algorithm). One needs a program/system to compress a file. Once compressed, to read it again, one needs a decoder/de-compressor.

Compressing data is not easy. One needs to maintain as much quality as possible. In today’s video encoders, for example, there are many options to compress in a certain system while having the ability to change a lot of variables. This power is give to you, the end user, because only you know at what point the quality will suffer – only you have access to the original file, and more importantly, only you know how to judge its aesthetic value.

What are the more mainstream encoding systems (Called Codecs)?

  • MPEG-1, 2 and 4 for Video
  • MPEG-3 or MP3 for audio
  • H.264 for Video
  • JPEG for Images
  • Prores, Cineform, DNxHD, Huffyuv, Lagarith, etc for video
  • FLAC, AC-3, PCM, etc for audio

For a comparison of video codecs, click here.

Each codec has its strengths and weaknesses. It is for the user to select the best system for his/her particular job. Sometimes, more than one codec is used in a particular pipeline. There are no rules here. Experience or expert advise is the most precious aid when dealing with codecs.

Encoding:

The ‘first’ time data is compressed, or coded, it is said to be encoded. The reason ‘first’ is in quotes, is because this is not technically true. Every time data is manipulated, or coded, encoding takes place. However, in the video world, we distinguish between encoding and transcoding (further coding after the first encode – see below) just to keep things simpler. We do this because most cameras already code the data with inbuilt circuitry.

Instead, it has made it more complicated! Tough luck! You can call it whatever you want, but when you communicate with another professional, make sure you agree on your terms before beginning anything.

Transcoding:

I define transcoding as – whenever a file which is already coded, is decoded and re-coded using another system. E.g., Changing from MPEG-2 to MPEG-4, or from lossless audio to MP3, etc.

That brings me to the point that compression can be lossless or ‘lossy’.

Lossless and Lossy:

Lossless systems don’t change the original data in any way. It just ‘rearranges’ the data to its own file system and renames the extension. Lossless systems are the best, of course, but with the problem of large file sizes.

Lossy systems (even ‘visually lossless’ systems like Cineform, Prores, etc) always change the original data when encoding or transcoding. Once converted, the data in the new file system cannot be said to have the same properties any more. There is always some loss with these systems. Some of these systems, like Cineform, Prores, DNxHD etc., might increase the file size after transcoding. So it is up to the user to decide on which system to adopt. There are too many factors involved to have rules. Don’t fall into the marketing traps by vendors!

Container:

Let’s say we have to watch a movie which has the following data ‘chunks’:

1. Moving Imagery or video – MPEG-4

2. Audio – MP3

3. Subtitles – SRT

You all know that in order to play files, you need software – called a Player generically. It is easier for this player to function if all the different data chunks with different codecs were ‘contained’ in one file, with one set of properties.

Hence programmers created another file, to contain and control different file systems under one roof. This file system is called a Container file.

What are the most common container formats?

  • Audio Video Interleave AVI – by Microsoft
  • Quicktime MOV – by Apple
  • OGG, WMA for audio.
  • RMVB – for Read Audio and video
  • VOB – for DVD movies
  • MXF, MPEG TS (Transport Stream), MP4 – used by cameras to encode video
  • WMV or WMA – Advanced systems format – by Windows
  • Others, like 3GP, Matroska (*.mkv), DivX

For a comparison of container formats, click here.

All these container files are just a bigger ‘folder’ in which smaller ‘folders’ can be kept. Some of them are linked to the software they are built for, and might cause problems with another piece of software. E.g., a lot of video professionals might have experienced AVI vs MOV issues when dealing with both. It’s part of the commercialization of data – My data, your data, blah blah. The idea is if you adopt one system, then you are a slave to it. This is why I prefer open sourced containers, like –

A container to watch out for:

One of the new players on the block is Matroska, or MKV. It is open-source, which means it is free to use. It is more powerful than many ‘professional’ containers out there. It can incorporate the widest number of codecs – think of a ship that can carry anything you put into it. For more information, please visit the official Matroska website.

For a list of open source codecs and containers, click here.

I hope I have given a general round-up of the terminology that one needs to understand before venturing into the chaotic and muddy waters of digital file management. Please visit the links provided for further information. The world of codecs is ever changing, and it is important to keep oneself up to date at all times.