Yes I am referring to the mainstay common gif you’ll find all over the web at large sites like Google’s owned Tenor or Facebook’s owned giphy. Everyone’s favorite file format for sharing short animated snippets.

The gif as most people know it

As most people know gif, gif is an animated file format. You might have looked at gif files and thought wow these files are pretty large. Perhaps you looked at them and thought: wow, these pictures are low definition. But at the end of the day, when you think of gif, you probably think of it as the short animated file format.

However this use case is drastically different from what the folks who wrote gif to expected it to be used for. In this post we’ll dive into the anatomy of a gif file and discuss some of its funkier features along the way.

Note that this post is supposed to be a fun exploration of how to understand the gif format and some of its more esoteric features. If you want to actually learn how to parse gifs, I would recommend these resources:

• The W3 spec
• Matthew Flickinger’s: What’s in a gif?
• I found this guide from ntfs.com helpful for early parts too

During this time I actually made a barely compliant gif parser using these resources called awful-gif which will parse some gifs. I don’t recommend using it.

Anyway onto the post.

The history of gif

The gif file format was created in 1987 by Compuserve. Back in 1987 gif was a rather compact format! It used compression, and not just any compression, but LZW compression. Many older file formats (some made by Compuserve) used RLE (Run Length Encoding) which in many cases isn’t nearly as efficient. One of the big win factors for gif was its solid compression ratio and good color gamut (a full 256 colors, wow!).¹

Two years later an addendum to the gif file format was created (gif89a) which added many of the features we know and love today.

Via the gif89a spec we can get a quick summary of all of the features supported in gif89 vs gif87a.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

Appendix
A. Quick Reference Table.

Block Name                  Required   Label       Ext.   Vers.
Application Extension       Opt. (*)   0xFF (255)  yes    89a
Comment Extension           Opt. (*)   0xFE (254)  yes    89a
Global Color Table          Opt. (1)   none        no     87a
Graphic Control Extension   Opt. (*)   0xF9 (249)  yes    89a
Header                      Req. (1)   none        no     N/A
Image Descriptor            Opt. (*)   0x2C (044)  no     87a (89a)
Local Color Table           Opt. (*)   none        no     87a
Logical Screen Descriptor   Req. (1)   none        no     87a (89a)
Plain Text Extension        Opt. (*)   0x01 (001)  yes    89a
Trailer                     Req. (1)   0x3B (059)  no     87a

Unlabeled Blocks
Header                      Req. (1)   none        no     N/A
Logical Screen Descriptor   Req. (1)   none        no     87a (89a)
Global Color Table          Opt. (1)   none        no     87a
Local Color Table           Opt. (*)   none        no     87a

Graphic-Rendering Blocks
Plain Text Extension        Opt. (*)   0x01 (001)  yes    89a
Image Descriptor            Opt. (*)   0x2C (044)  no     87a (89a)

Control Blocks
Graphic Control Extension   Opt. (*)   0xF9 (249)  yes    89a

Special Purpose Blocks
Trailer                     Req. (1)   0x3B (059)  no     87a
Comment Extension           Opt. (*)   0xFE (254)  yes    89a
Application Extension       Opt. (*)   0xFF (255)  yes    89a

legend:           (1)   if present, at most one occurrence
                  (*)   zero or more occurrences
                  (+)   one or more occurrences

Much of this is going to be gibberish to folks who haven’t read the whole spec before, so let’s discuss how a gif is put together a bit, and we’ll talk about some of its oddities along the way.

Some fun before we begin from the spec:

1
2
3
4
5

Appendix
D. Conventions.

Animation - The Graphics Interchange Format is not intended as a platform for
animation, even though it can be done in a limited way.

Anyway let’s begin ;)

The anatomy of a gif

I’m going to walk through this with an example, so if you’d like to follow along, feel free! Right click and download and you’re good to go.

²

If you’re following along at home, all you need is a machine with a hexdump tool installed. I’ll be using xxd which is preinstalled on most unixes (Linux, macOS), or can be installed with the package vim-common.

The gif header

Every gif starts with a header where the magic bits signifying what what type of gif it is and a little extra information giving basic details about the image.

1
2

🜛 xxd Sunflower_as_gif_websafe_89a.gif | head -1 # and some arrows
00000000: -> 4749 4638 3961 <- dc00 0501 f700 0002 0102  GIF89a..........

xxd makes the work easy for us for the first few bytes and tries to decode bytes to ascii if it makes sense. Take a look at that, GIF89a! It’s a certified valid gif!

Each letter stands out as one byte, so the magic bytes we’re looking for here are: 0x47, 0x49, 0x46, 0x38, 0x39, 0x61.

Optionally last three bytes may be: 0x38, 0x37, 0x61 if only supporting the gif87a file format. We won’t go into the older version of the format as much as gif89.

There isn’t much else interesting in the header since it’s just static bytes, so let’s keep moving on.

Detour: Hey who accepts gif87a?

While looking into gifs I wanted to see if either of the major gif hosting providers would accept and retain gif87a. Would they work, or just err out?

Here’s a gif87a version of the sunflower we were looking at earlier. The 87a version of this image will only be used for this section.

Let’s upload the image to 4 of the major gif hosting providers:

• tenor
• giphy
• imgur
• gfycat

This is what we start with:

1
2

🜛 xxd Sunflower_as_gif_websafe_gif87a.gif | head -1
00000000: 4749 4638 3761 fa00 2901 f500 00ff cc33  GIF87a..)......3

Here are the results after redownloading the image I just uploaded.

Tenor re-encodes to gif89a:

1
2

🜛 Downloads xxd tenor.gif | head -1
00000000: 4749 4638 3961 a401 f201 f700 0006 0406  GIF89a..........

giphy re-encodes to a gif89a:

1
2

🜛 Downloads xxd giphy.gif | head -1
00000000: 4749 4638 3961 fa00 2901 f525 0000 0000  GIF89a..)..%....

Actually that’s a little disingenuous, giphy ONLY accepts animated gifs, so we have to click edit (which shows the frame editor) and click done. Multiple images are allowed to be stored in the gif87a spec, but they cannot have a delay (hence no animation³). I guess I’m not sure what I expected here.

imgur preserves the original file!!!

1
2

🜛 Downloads xxd aUxm3NN.gif | head -1
00000000: 4749 4638 3761 fa00 2901 f500 00ff cc33  GIF87a..)......3

As for gfycat, well it’s been stuck in the last phase of “encoding” the for last 20 minutes. Hopefully I didn’t make an alert for a poor engineer on the weekend.

This short analysis shows the two biggest hosting providers made or owned by two of the largest tech companies in the world don’t respect my old gif and completely rewrite it. In fact for a company called giphy it only seems to respect one kind of gif. I’ll have to have a talk with the giphy team on Tuesday…

Anyway back to exploring the file format.

The logical screen descriptor

Hey how does your image show up in a certain resolution? Say we use the “get info” function in preview on macOS, how does it know this image is 220x261?

Well believe it or not, that’s built into the file format!⁴

Bytes 0x6-0xA have this and a little more info in it. Bytes 0x6 and 0x8 refer to the length and width.

1
2

🜛 xxd Sunflower_as_gif_websafe_89a.gif | head -1 # and some arrows
00000000:  4749 4638 3961 -> dc00 0501 <- f700 0002 0102  GIF89a..........

Each dimension gets two bytes to specify size. Also it’s important to remember that all bytes in gif file format are specified as little endian⁵.

First comes width, which is 0x00dc (reordered from dc00) => 220 in decimal

Then comes length which is 0x0105 (reordered from 0501) => 261 in decimal

Detour: Does that mean we have a resolution limit on our gifs?

That’s right! Since we only get two bytes each, no one resolution, width or length can be larger than 65535. We can confirm that by trying to make a new gif of 1x65536 in gimp:

Other file formats don’t fall short to this regard. If you’d like to download the widest png in existence you may do so here. It’s a small download, but may crash your image viewer if you open it. Firefox struggles to open it and says there is an error even though it’s spec compliant.

Back to the logical screen descriptor

The logical screen descriptor isn’t done yet though, next come a set of packed fields. It’s easier to explain with the diagram from the spec:

1
2
3
4

     <Packed Fields>  =      Global Color Table Flag       1 Bit
                             Color Resolution              3 Bits
                             Sort Flag                     1 Bit
                             Size of Global Color Table    3 Bits

This has information on the global color table which will come after the logical screen descriptor if the global color table bit is set.

Color resolution decides how many bytes there are per color in the global color table.

The sort flag is supposed to be a flag that tells the decoder earlier colors are more important by sorting colors in a most to least useful fashion.

And the size of the global color table is, well, how big the color table is.

In our sunflower at byte 0xA, we have the result 0xF7

1
2

🜛 xxd Sunflower_as_gif_websafe_89a.gif | head -1
00000000: 4749 4638 3961 dc00 0501 -> f7 <- 00 0002 0102  GIF89a..........

Or in binary that is: 1111 0111

That basically means we have a fully loaded gif except that the GCT is not sorted.

1
2
3
4
5
6
7
8
9

                             ┌──────────GCT not sorted
                             ▼          by importance
                        1111 0111
                        ▲───  ───
      GCT set───────────┘ ▲    ▲
                          │    │
    3 bytes per           │    └─────GCT is 768 bytes
    color    ─────────────┘          (max size)
(max resolution)

I suppose we’ve got this far and haven’t even discussed what the global color table is. The global color table holds the colors used in each section of bytes. They’re standard RGB values from 0-255 you could plug into any modern RGB color picker.

Detour: Hey hold on, is that global color table optional?

If you had a sharp eye, you might have noticed that the first bit in byte 0xA says the GCT can be optional. Well that’s interesting. How do we render an image without it specifying what colors it needs?

According to the spec:

Color Tables - Both color tables, the Global and the Local, are optional; if present, the Global Color Table is to be used with every image in the Data Stream for which a Local Color Table is not given; if present, a Local Color Table overrides the Global Color Table. However, if neither color table is present, the application program is free to use an arbitrary color table.

Well that’s neat. Hey what if we remove the global color table to an image, what might modern renders do with our image? Something amazing I’m sure.

Our image specifies color table size is 768 bytes. It starts on byte 0xA… say we just zero out the most significant bit of byte 0xA like so.

And then delete until byte 789 (exclusive):

1
2

🜛 xxd Sunflower_as_gif_89a-no-gct.gif | head -1
00000000: 4749 4638 3961 dc00 0501 007f 8121 f904  GIF89a.......!..

Now the first line ends like this which is still a perfectly valid gif. How might that look?

Amazing! Stupendous! Wonderful! As of the time of this writing, it’s just a perfectly black square. And this is the case in every single renderer I’ve tried. Gimp, Chrome, Firefox, Preview, gifiddle, you name it.

That’s a little boring, but I’m not sure what I expected.

Anyway back to the logical screen descriptor

The logical screen descriptor continued

After the bytes that describe the global color table, there are two final bytes describing the screen descriptor.

Byte B is the background color which refers to an index into the global color table, and byte C is the pixel aspect ratio, describing the squareness of the pixel.

1
2
3
4
5
6
7
8

🜛 xxd Sunflower_as_gif_websafe_89a.gif | head -1
00000000: 4749 4638 3961 dc00 0501 f700 0002 0102  GIF89a..........
                                     ^  ^
                                     |  |
Background color is color in index 0 of |
GCT                                     |
                                        Pixel aspect ratio is 0:0 or host
                                        pixel aspect ratio.

Detour: Hey wait a second, pixel aspect ratio, what’s that?

Pixels weren’t always square! Also bytes weren’t always 8 bits, but that’s a tangent I won’t go into.

Gif, and actually some of the other most popular modern image formats support non-square pixels.

Hey! I wonder how compliant our most popular gif renderers are when rendering non-square pixels. Probably very compliant. Let’s try a popular test in Firefox and Chrome and see how they look: http://frs.badcoffee.info/PAR_AcidTest/

Uhh…. well that’s something. Those are in order: jpg, png, and gif. And Firefox, Chrome, and Preview all ignore the aspect ratio.

Unfortunately this is widely unsupported and there is currently a 16 year old bug in Firefox for it: https://bugzilla.mozilla.org/show_bug.cgi?id=333377

Even Gifiddle, the most compliant gif viewer I’ve found doesn’t support non-square pixels: https://github.com/ata4/gifiddle/issues/1

We’ll discuss gifiddle more as we try out more obscure gif features.

If you really want to display non-square pixels, you can sort of massage gimp to do it. Also grafx2 can apparently handle very specific odd resolutions of pixels. I have not tested it myself though.

Onto the global color table

The global color table (GCT) is easily the most boring part of the gif. You jump through the image in multiples of 3 from 0 to the size of the global color table. There really isn’t anything worth talking about here.

The best way for me to illustrate this is to point to my awful-gif project and it’ll output all of the colors in the GCT of the sunflower (and maybe other images too).

The GCT parsing is right here and you can see there really isn’t anything special about it.

Run with:

1

cargo run --quiet -- --gif-file ./experiments/Sunflower_as_gif_websafe.gif

The optional graphic control extension

Now we have the graphic control extension (GCE), introduced by the extension introducer: 0x21 (extension introduced) and then 0xF9 (!)

There are a number of extensions, but the graphic control extension is arguably one of the most important at least in modern day usage. The GCE is required to make gifs “animated” by allowing a delay time between frames, among other things.

1
2
3

🜛 xxd Sunflower_as_gif_websafe_89a.gif | head -50 | tail -2
00000300: 88ae b091 a5b1 a4b9 be94 887f 81 -> 21 f904  .............!..
00000310: 0000 0000 <- 0021 fe51 4669 6c65 2073 6f75  .....!.QFile sou

This gif isn’t exactly animated, so there isn’t a lot going on here. Lots of zeroes as you can see, but we’ll still go into each byte.

The first byte is the block size, which in this case is 0x04, but actually according to the spec is always 0x04.

Detour: Hold on just a second, can we get rid of the block size then?

Ah if the block size is always a static constant, it isn’t really important then, is it? Technically it’s part of the spec, but it doesn’t actually do anything. Let’s play our favorite game of opening it in the most popular image viewers.

For these tests I’ll be using a much simpler gif to make it easier to see what happens:

For the following tests I’ve modified it to remove the GCE. The modified version is kept in xxd format below. To can reassemble it:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

00000000: 4749 4638 3961 2000 3400 f0ff 00ff ffff  GIF89a .4.......
00000010: 0000 0021 f903 0500 0002 002c 0000 0000  ...!.......,....
00000020: 2000 3400 0002 788c 8fa9 cb0b 0fa3 94ed   .4...x.........
00000030: cc7b abc1 1cea d075 5fc8 8d64 a69d 68a5  .{.....u_..d..h.
00000040: 4e66 eba5 702c 3675 cddc a5bd e34e bfcb  Nf..p,6u.....N..
00000050: 0131 ace1 ea47 0405 9128 9f42 9714 2667  .1...G...(.B..&g
00000060: a70d 3564 bd1a b52e 25b7 f905 8729 de31  ..5d....%....).1
00000070: cd1c c9a2 016a 74db fc1e c7c3 f36f 9d7b  .....jt......o.{
00000080: d7e6 af7b 6a7f f607 13d8 32a8 5258 55e6  ...{j.....2.RXU.
00000090: 9608 b728 d748 f768 1789 f751 b950 0000  ...(.H.h...Q.P..
000000a0: 3b                                       ;

save it to a text file called invalid.hex and do: xxd -r invalid.hex > invalid.gif

(If you can’t find the updated byte, it’s at byte: 0x16 and changed from 0x4 -> 0x03)

First macOS Preview:

Preview being standards compliant! We love to see it! Even if it technically doesn’t matter for a file format that will probably never get an update.

Next let’s try Firefox:

Firefox knows it’s a static value and ignores the result of it. Not exactly standards compliant, but probably the smartest thing to do.

Chrome goes a little bananas when the block size is removed. Not exactly inverting it, it does… something which I’m not certain. Chrome is most certainly the least compliant with the standards here.

Back to the graphics control extension

After we read past the block size, we then get a packed field described as:

1
2
3
4

      <Packed Fields>  =     Reserved                      3 Bits
                             Disposal Method               3 Bits
                             User Input Flag               1 Bit
                             Transparent Color Flag        1 Bit

All of these fields are set to 0 in our image, so I will just explain them.

Reserved is set for when gif22a comes out and we need those three bits for something good.

User input is for taking user input to advance a gif to the next image with a mouse click or keyboard press.

The transparent index is for setting whether or not we should allow transparency.

Detour: WAIT A SECOND GIFS CAN TAKE USER INPUT ???

Yeah you read that right. Gifs can take user input to advance to the next frame. This poor guy built a whole site around recreating this feature with pngs. A shame he didn’t read the gif spec after being stuck inside for 2 years like me.

While we’re in the weird features section of gif, we might as well discuss the other weird feature gif supports, the plain text extension.

The plain text extension allows the gif maker to embed monospace text wherever they like with some basic styling directly on the image.

The plain text extension like the user input extension, was likely never implemented by any gif viewer besides ones made by eccentric folk for fun like the guy who made gifiddle.

BOB_89A.gif, likely the first gif ever posted on the internet is an example of a gif that uses both.

Here BOB_89A.gif rendered in a modern browser (your own browser, if it’s not modern please update it):

However if you put it into gifiddle, you get a very different result, with the last message being a very important truth.

I won’t spoil the surprise though, give it a right click download and then put that gif into gifiddle to see what happens.

Gifiddle link: http://ata4.github.io/gifiddle/

Neither of these features are supported by any modern browser or gif viewer.

If you’d like to read more about the plain text extension, you can do so here.

The optional comment extension

The comment extension to come next, and actually may appear anywhere a block may begin. However it most often appears in this part of a gif.

The comment section is only allowed to contain 7 bit ascii and is intended for humans to read.

Since the comment section is just ascii, you can just fire off strings and find the comment in the output:

1
2

🜛 strings Sunflower_as_gif_websafe_89a.gif | head -7 | tail -1
QFile source: https://commons.wikimedia.org/wiki/File:Sunflower_as_gif_websafe.gif

In this image it starts around byte 0x310 in our image:

1
2
3
4
5
6
7

🜛 xxd Sunflower_as_gif_websafe_89a.gif | head -55 | tail -6
00000310: 0000 0000 0021 fe51 4669 6c65 2073 6f75  .....!.QFile sou
00000320: 7263 653a 2068 7474 7073 3a2f 2f63 6f6d  rce: https://com
00000330: 6d6f 6e73 2e77 696b 696d 6564 6961 2e6f  mons.wikimedia.o
00000340: 7267 2f77 696b 692f 4669 6c65 3a53 756e  rg/wiki/File:Sun
00000350: 666c 6f77 6572 5f61 735f 6769 665f 7765  flower_as_gif_we
00000360: 6273 6166 652e 6769 6600 2c00 0000 00dc  bsafe.gif.,.....

Signified again by a BANG (!) (extension introduced) and 0xfe comment extension. Afterwards the comment should be read for 255 bytes or until a 0x00 is read.

The rest of the image data

There isn’t much more to talk about after that. This image skips out on most other gif features like the local color table and animations, so the majority of the rest of the gif is just data and terminators.

I’ll be perfectly honest, lzw compression isn’t terribly difficult to learn, but explaining it is not the purpose of this blogpost. If you’d like to learn it, Matthew Flickinger has a great post on his site about it.

Bonus section: True color gifs

Did you know that gifs can be true color? It requires a little insanity, but if you remember the world “local color table” then it might make sense. Each data segment is allowed to have its own local color table, and thus if you break a gif up into enough pieces, you get true color!

Most gifs don’t do this for a couple of reasons.

First, the resulting image will be gigantic. Each new palette of 256 colors will consume an additional 768 bytes.

Second, renderers nowadays will not “properly” render the image. Browsers by default will often put a delay of 0.1 between frames if not specified.

However a truly compliant gif renderer will properly display a truecolor gif. So if you have the space, and the memory, and the spare CPU for it, why not have a true color gif? We can all have a little true color as a treat.

If you’d like more information on truecolor gifs, Wikipedia has a whole section on it called True color.

Wrapping things up

Anyway if you made it this far, I appreciate it. This was much longer than my average post here. But there was a lot to say!

In fact there’s really a lot more to say too. There are more parts of the gif spec I didn’t go over, and nuances to the format that I could, I dunno, write a whole spec about.

If you’re interested in learning more about gif, I recommend checking out the spec and all of the other links I added at the top of the post.

Otherwise, thanks for reading :)