Yes I am referring to the mainstay common gif you’ll find all over the web at large sites like Google’s owned Tenor or Facebook’s owned giphy. Everyone’s favorite file format for sharing short animated snippets.
The gif as most people know it
As most people know gif, gif is an animated file format. You might have looked at gif files and thought wow these files are pretty large. Perhaps you looked at them and thought: wow, these pictures are low definition. But at the end of the day, when you think of gif, you probably think of it as the short animated file format.
However this use case is drastically different from what the folks who wrote gif to expected it to be used for. In this post we’ll dive into the anatomy of a gif file and discuss some of its funkier features along the way.
Note that this post is supposed to be a fun exploration of how to understand the gif format and some of its more esoteric features. If you want to actually learn how to parse gifs, I would recommend these resources:
- The W3 spec
- Matthew Flickinger’s: What’s in a gif?
- I found this guide from ntfs.com helpful for early parts too
During this time I actually made a barely compliant gif parser using these resources called awful-gif which will parse some gifs. I don’t recommend using it.
Anyway onto the post.
The history of gif
The gif file format was created in 1987 by Compuserve. Back in 1987 gif was a rather compact format! It used compression, and not just any compression, but LZW compression. Many older file formats (some made by Compuserve) used RLE (Run Length Encoding) which in many cases isn’t nearly as efficient. One of the big win factors for gif was its solid compression ratio and good color gamut (a full 256 colors, wow!).1
Two years later an addendum to the gif file format was created (gif89a) which added many of the features we know and love today.
Via the gif89a spec we can get a quick summary of all of the features supported in gif89 vs gif87a.
Much of this is going to be gibberish to folks who haven’t read the whole spec before, so let’s discuss how a gif is put together a bit, and we’ll talk about some of its oddities along the way.
Some fun before we begin from the spec:
Anyway let’s begin ;)
The anatomy of a gif
I’m going to walk through this with an example, so if you’d like to follow along, feel free! Right click and download and you’re good to go.
If you’re following along at home, all you need is a machine with a hexdump tool installed. I’ll be using xxd which is preinstalled on most unixes (Linux, macOS), or can be installed with the package vim-common.
The gif header
Every gif starts with a header where the magic bits signifying what what type of gif it is and a little extra information giving basic details about the image.
xxd makes the work easy for us for the first few bytes and tries to decode bytes to ascii if it makes sense. Take a look at that, GIF89a! It’s a certified valid gif!
Each letter stands out as one byte, so the magic bytes we’re looking for here
0x47, 0x49, 0x46, 0x38, 0x39, 0x61.
Optionally last three bytes may be:
0x38, 0x37, 0x61 if only
supporting the gif87a file format. We won’t go into the older version of the
format as much as gif89.
There isn’t much else interesting in the header since it’s just static bytes, so let’s keep moving on.
Detour: Hey who accepts gif87a?
While looking into gifs I wanted to see if either of the major gif hosting providers would accept and retain gif87a. Would they work, or just err out?
Here’s a gif87a version of the sunflower we were looking at earlier. The 87a version of this image will only be used for this section.
Let’s upload the image to 4 of the major gif hosting providers:
This is what we start with:
Here are the results after redownloading the image I just uploaded.
Tenor re-encodes to gif89a:
giphy re-encodes to a gif89a:
Actually that’s a little disingenuous, giphy ONLY accepts animated gifs, so we have to click edit (which shows the frame editor) and click done. Multiple images are allowed to be stored in the gif87a spec, but they cannot have a delay (hence no animation3). I guess I’m not sure what I expected here.
imgur preserves the original file!!!
As for gfycat, well it’s been stuck in the last phase of “encoding” the for last 20 minutes. Hopefully I didn’t make an alert for a poor engineer on the weekend.
This short analysis shows the two biggest hosting providers made or owned by two of the largest tech companies in the world don’t respect my old gif and completely rewrite it. In fact for a company called giphy it only seems to respect one kind of gif. I’ll have to have a talk with the giphy team on Tuesday…
Anyway back to exploring the file format.
The logical screen descriptor
Hey how does your image show up in a certain resolution? Say we use the “get info” function in preview on macOS, how does it know this image is 220x261?
Well believe it or not, that’s built into the file format!4
Bytes 0x6-0xA have this and a little more info in it. Bytes 0x6 and 0x8 refer to the length and width.
Each dimension gets two bytes to specify size. Also it’s important to remember that all bytes in gif file format are specified as little endian5.
First comes width, which is 0x00dc (reordered from dc00) => 220 in decimal
Then comes length which is 0x0105 (reordered from 0501) => 261 in decimal
Detour: Does that mean we have a resolution limit on our gifs?
That’s right! Since we only get two bytes each, no one resolution, width or length can be larger than 65535. We can confirm that by trying to make a new gif of 1x65536 in gimp:
Other file formats don’t fall short to this regard. If you’d like to download the widest png in existence you may do so here. It’s a small download, but may crash your image viewer if you open it. Firefox struggles to open it and says there is an error even though it’s spec compliant.
Back to the logical screen descriptor
The logical screen descriptor isn’t done yet though, next come a set of packed fields. It’s easier to explain with the diagram from the spec:
This has information on the global color table which will come after the logical screen descriptor if the global color table bit is set.
Color resolution decides how many bytes there are per color in the global color table.
The sort flag is supposed to be a flag that tells the decoder earlier colors are more important by sorting colors in a most to least useful fashion.
And the size of the global color table is, well, how big the color table is.
In our sunflower at byte 0xA, we have the result 0xF7
Or in binary that is:
That basically means we have a fully loaded gif except that the GCT is not sorted.
I suppose we’ve got this far and haven’t even discussed what the global color table is. The global color table holds the colors used in each section of bytes. They’re standard RGB values from 0-255 you could plug into any modern RGB color picker.
Detour: Hey hold on, is that global color table optional?
If you had a sharp eye, you might have noticed that the first bit in byte 0xA says the GCT can be optional. Well that’s interesting. How do we render an image without it specifying what colors it needs?
According to the spec:
Color Tables - Both color tables, the Global and the Local, are optional; if present, the Global Color Table is to be used with every image in the Data Stream for which a Local Color Table is not given; if present, a Local Color Table overrides the Global Color Table. However, if neither color table is present, the application program is free to use an arbitrary color table.
Well that’s neat. Hey what if we remove the global color table to an image, what might modern renders do with our image? Something amazing I’m sure.
Our image specifies color table size is 768 bytes. It starts on byte 0xA… say we just zero out the most significant bit of byte 0xA like so.
And then delete until byte 789 (exclusive):
Now the first line ends like this which is still a perfectly valid gif. How might that look?
Amazing! Stupendous! Wonderful! As of the time of this writing, it’s just a perfectly black square. And this is the case in every single renderer I’ve tried. Gimp, Chrome, Firefox, Preview, gifiddle, you name it.
That’s a little boring, but I’m not sure what I expected.
Anyway back to the logical screen descriptor
The logical screen descriptor continued
After the bytes that describe the global color table, there are two final bytes describing the screen descriptor.
Byte B is the background color which refers to an index into the global color table, and byte C is the pixel aspect ratio, describing the squareness of the pixel.
Detour: Hey wait a second, pixel aspect ratio, what’s that?
Pixels weren’t always square! Also bytes weren’t always 8 bits, but that’s a tangent I won’t go into.
Gif, and actually some of the other most popular modern image formats support non-square pixels.
Hey! I wonder how compliant our most popular gif renderers are when rendering non-square pixels. Probably very compliant. Let’s try a popular test in Firefox and Chrome and see how they look: http://frs.badcoffee.info/PAR_AcidTest/
Uhh…. well that’s something. Those are in order: jpg, png, and gif. And Firefox, Chrome, and Preview all ignore the aspect ratio.
Unfortunately this is widely unsupported and there is currently a 16 year old bug in Firefox for it: https://bugzilla.mozilla.org/show_bug.cgi?id=333377
We’ll discuss gifiddle more as we try out more obscure gif features.
Onto the global color table
The global color table (GCT) is easily the most boring part of the gif. You jump through the image in multiples of 3 from 0 to the size of the global color table. There really isn’t anything worth talking about here.
The best way for me to illustrate this is to point to my awful-gif project and it’ll output all of the colors in the GCT of the sunflower (and maybe other images too).
The GCT parsing is right here and you can see there really isn’t anything special about it.
The optional graphic control extension
Now we have the graphic control extension (GCE), introduced by the extension introducer: 0x21 (extension introduced) and then 0xF9 (!)
There are a number of extensions, but the graphic control extension is arguably one of the most important at least in modern day usage. The GCE is required to make gifs “animated” by allowing a delay time between frames, among other things.
This gif isn’t exactly animated, so there isn’t a lot going on here. Lots of zeroes as you can see, but we’ll still go into each byte.
The first byte is the block size, which in this case is 0x04, but actually according to the spec is always 0x04.
Detour: Hold on just a second, can we get rid of the block size then?
Ah if the block size is always a static constant, it isn’t really important then, is it? Technically it’s part of the spec, but it doesn’t actually do anything. Let’s play our favorite game of opening it in the most popular image viewers.
For these tests I’ll be using a much simpler gif to make it easier to see what happens:
For the following tests I’ve modified it to remove the GCE. The modified version is kept in xxd format below. To can reassemble it:
save it to a text file called invalid.hex and do:
xxd -r invalid.hex > invalid.gif
(If you can’t find the updated byte, it’s at byte: 0x16 and changed from 0x4 -> 0x03)
First macOS Preview:
Preview being standards compliant! We love to see it! Even if it technically doesn’t matter for a file format that will probably never get an update.
Next let’s try Firefox:
Firefox knows it’s a static value and ignores the result of it. Not exactly standards compliant, but probably the smartest thing to do.
Chrome goes a little bananas when the block size is removed. Not exactly inverting it, it does… something which I’m not certain. Chrome is most certainly the least compliant with the standards here.
Back to the graphics control extension
After we read past the block size, we then get a packed field described as:
All of these fields are set to 0 in our image, so I will just explain them.
Reserved is set for when gif22a comes out and we need those three bits for something good.
User input is for taking user input to advance a gif to the next image with a mouse click or keyboard press.
The transparent index is for setting whether or not we should allow transparency.
Detour: WAIT A SECOND GIFS CAN TAKE USER INPUT ???
Yeah you read that right. Gifs can take user input to advance to the next frame. This poor guy built a whole site around recreating this feature with pngs. A shame he didn’t read the gif spec after being stuck inside for 2 years like me.
While we’re in the weird features section of gif, we might as well discuss the other weird feature gif supports, the plain text extension.
The plain text extension allows the gif maker to embed monospace text wherever they like with some basic styling directly on the image.
The plain text extension like the user input extension, was likely never implemented by any gif viewer besides ones made by eccentric folk for fun like the guy who made gifiddle.
BOB_89A.gif, likely the first gif ever posted on the internet is an example of a gif that uses both.
Here BOB_89A.gif rendered in a modern browser (your own browser, if it’s not modern please update it):
However if you put it into gifiddle, you get a very different result, with the last message being a very important truth.
I won’t spoil the surprise though, give it a right click download and then put that gif into gifiddle to see what happens.
Gifiddle link: http://ata4.github.io/gifiddle/
Neither of these features are supported by any modern browser or gif viewer.
If you’d like to read more about the plain text extension, you can do so here.
The optional comment extension
The comment extension to come next, and actually may appear anywhere a block may begin. However it most often appears in this part of a gif.
The comment section is only allowed to contain 7 bit ascii and is intended for humans to read.
Since the comment section is just ascii, you can just fire off strings and find the comment in the output:
In this image it starts around byte 0x310 in our image:
Signified again by a BANG (!) (extension introduced) and 0xfe comment extension. Afterwards the comment should be read for 255 bytes or until a 0x00 is read.
The rest of the image data
There isn’t much more to talk about after that. This image skips out on most other gif features like the local color table and animations, so the majority of the rest of the gif is just data and terminators.
I’ll be perfectly honest, lzw compression isn’t terribly difficult to learn, but explaining it is not the purpose of this blogpost. If you’d like to learn it, Matthew Flickinger has a great post on his site about it.
Bonus section: True color gifs
Did you know that gifs can be true color? It requires a little insanity, but if you remember the world “local color table” then it might make sense. Each data segment is allowed to have its own local color table, and thus if you break a gif up into enough pieces, you get true color!
Most gifs don’t do this for a couple of reasons.
First, the resulting image will be gigantic. Each new palette of 256 colors will consume an additional 768 bytes.
Second, renderers nowadays will not “properly” render the image. Browsers by default will often put a delay of 0.1 between frames if not specified.
However a truly compliant gif renderer will properly display a truecolor gif. So if you have the space, and the memory, and the spare CPU for it, why not have a true color gif? We can all have a little true color as a treat.
If you’d like more information on truecolor gifs, Wikipedia has a whole section on it called True color.
Wrapping things up
Anyway if you made it this far, I appreciate it. This was much longer than my average post here. But there was a lot to say!
In fact there’s really a lot more to say too. There are more parts of the gif spec I didn’t go over, and nuances to the format that I could, I dunno, write a whole spec about.
If you’re interested in learning more about gif, I recommend checking out the spec and all of the other links I added at the top of the post.
Otherwise, thanks for reading :)
Sunflower rehosted from Wikipedia article on gifs (see footnote #1) ↩︎
Ok gif87a technically supports animation in a more limited format. But I’ve already written almost 3500 words at the time of this footnote and I don’t want to get into every detail. For more information you can try out the gif87a animation examples on the gifiddle repo: https://github.com/ata4/gifiddle ↩︎
For more information, see section 18 (Logical screen descriptor) of the gif spec. ↩︎