I-frames & P-frames

Fleshing out my initial understanding of the basics of datamoshing…

Digital video files are compressed in order to save file space. Digital video is ultimately pixel-level color information (i.e. pixel #3 is RGB color 255,0,0 at frame #45). In a video, there are times when a pixel’s color remains static over a period of time (i.e. in a video of a bird flying across a solid blue sky, the blue sky does not change). To save space, we can skip saving data about pixels whose color does not change.

This (I think) is how it’s done: The initial frame of a new video is stored as an I-Frame. An I-Frame contains color information for every single pixel on the screen. Then, every subsequent frame of the same video is composed of P-frames. P-frames contain information only about what has changed since the previous frame. P-frames contain motion information.

I-Frames

• Exist independently
• Contains color information for every single pixel on the screen
• i.e.: [255,0,244,233,2,123,23,64]

P-Frames

• Rely on a previous/existing I-Frame
• Contains motion information for only the pixels that change
• i.e.: [Move block from (x1,y1) to (x2,y2); change color slightly in region X]

In a video, the frames would look like: [I-frame, p-frame, p-frame, p-frame, I-frame, p-frame, p-frame, p-frame]

In a video with two distinct scenes, with one cut in the middle, there would be two I-Frames. The first I-Frame occurs at the very first frame of the video. The second I-Frame occurs at the first frame after the cut. Since the second scene is different, we need a new I-Frame for the subsequent P-Frames to follow.

One form of datamoshing is performed when you remove that second I-Frame. What happens is the second set of P-frames alter the “incorrect” I-Frame. The effect you get is motion information being applied to the “wrong” pixels. The example below is unfortunately suggestive and may be offensive to some viewers.

Other forms of datamoshing involve manipulating the motion data directly. This is exciting for a newbie like me because it feels like manipulating down at the pixel-level opposed to pre-made transitions and effects (which I guess in a way also happen at the pixel-level(?) + I am using a pre-made After Effects plugin).

It will be a fun challenge to explore the narrative storytelling possibilities of datamoshing. For now, I feel much of its appeal is how interesting it is on a “technical” level. To an outside audience it just looks like video vomit. Initially, I feel it has applications for absurd comedy, satire, and parody… the effect or idea that the video medium itself is breaking down…