How Math Broke Media

The Underlying Logic of Media Will Change in Every Conceivable Way

Image Source: Nano Banana.

“In a sea change, nothing is safe”

—Beck, Little One

It’s easy to get stuck in your thinking. The prefrontal cortex uses a lot of calories, so we evolved to operate on autopilot when we can. Habits, assumptions, conventions, and even the words we use carve deep cognitive grooves, allowing us to function without thinking too much. The more you are immersed, the deeper those grooves and the harder it is to see out.

That’s okay when things are stable, but not when things are changing—especially when many components of a complex system are changing at the same time, like now.

With a little hindsight, we can see this myopia at work when a new technological paradigm emerges. We look back now and chuckle at things that seem so obvious today, but were apparently not so obvious then. The first cars were called “horseless carriages” and were designed like carriages, albeit without the horse part. When electricity became widely accessible, factories used electric lights to extend the workday so that they could run their steam engines longer, rather than electrify the production process. In media, the first applications of new mediums are often just imitations of prior media, or what is called skeuomorphism. The first online magazines were little more than static pdfs; the first films were recorded stage plays; the first radio shows were broadcasts of Vaudeville acts. It takes a while for creatives to figure out how to exploit the unique attributes of the new medium.

Over the last three years, I’ve been trying to convey how profoundly AI will change the media business. This post is another try. It offers a unifying framework to put the current moment in perspective and hopefully help people get unstuck.

Tl;dr:

• GenAI is not an isolated breakthrough. It is the culmination of a forty-year process: the mathematization of media. It is causing a shift from a world where media had a continuous substrate (i.e., underlying layer) and discrete businesses to one where the substrate is discrete and the business becomes continuous.

• Before digitization, media signals were analog and continuous. They are largely incompatible, so each media format evolved in its own parallel, discrete silo.

• Digitization made the substrate discrete, because it converted all media to bits. It also created a new symbolic layer, because bits are universal representative symbols. This enabled the convergence of all media on one network and a few form factors, which made distribution continuous.

• As a result, old distinctions between formats, geographies, economic models, regulatory frameworks, and support structures are all eroding. This is the cause of much of the displacement in the media business today.

• GenAI is the next step in this process. It introduces a second symbolic layer—a semantic layer, comprising combinatorial symbols. In a multimodal, continuous latent space, any element can be combined and manipulated cheaply and reversibly. That will make creation continuous.

• Distinctions between workflows and labor roles, stages of production, creation and distribution, and creator and consumer will eventually also fall away.

• This is poised to change the underlying logic of media in every conceivable way over time: how it is defined, developed, created, distributed, marketed, funded, and monetized.

• “Eventually” and “over time” are important qualifiers. The timing matters a lot. What I’m describing will play out over years and decades, not tomorrow. But the long term arc of technology is clear.

• Even those who believe they are thinking about the implications expansively are probably not thinking big enough.

---

---

Era 1 – The Pre-Math World (Continuous Substrate = Discrete Businesses)

For the first 500 years or so of mass media¹, the substrate (i.e., underlying layer) was continuous and the businesses were discrete.

Let’s start with the substrate. From Infinite Content, Chapter 3:

All media are representations of sound, images and/or text. In nature, sound and light waves occur over a continuous range of values. (Text is a special case. It doesn’t occur in nature, since it’s an abstraction of language, which is in turn an abstraction of human thought.)

Let’s take sound, to make it more concrete. Sound occurs when something happens that makes an object vibrate, like vocal cords or a string. The vibration creates a pressure wave—a continuous signal—that moves through the air. If it is powerful enough, occurs in the right frequency range and reaches a receiver, like ear drums or a microphone, it produces sound…

…Prior to the commercialization of digital technologies, to reproduce sounds and images, most media had to also replicate this continuous range of values. Media was analog, so called because it was analogous to the source signal.

Analog forms are fundamentally incompatible, so there is no shared language between them. They lack universality. Sound exists as pressure waves in the air; images exist as photon distribution; text exists as shapes of ink on a screen or paper.

Much like Darwin’s finches, the incompatibility between the physical layers of each medium caused each to evolve within its own distinct ecosystem:

• Distribution infrastructures. Movies were screened in movie theaters, projected on big screens by specialized cinema projectors. Books, magazines, records and VHS tapes were distributed like other packaged goods, through local distributors, rack jobbers, and, ultimately, (“brick and mortar”) retailers. Newspapers are far more ephemeral and time sensitive than most other media, so they require a highly specialized network of local distributors to get them to newsstands and people’s homes every day. Radio and TV were both broadcast over radio frequency spectrum, but the frequencies and amount of spectrum they used was tailored to their specific characteristics.²

• Form factors. Each medium had its own specialized form factor. You read the news in a newspaper; watched TV on a TV; listened to the radio on a radio, etc.

• Economic models. Each media format had a discrete monetization logic too. Let’s take movies, which had (and sometimes still have) a precise and specific pricing and windowing framework. A movie produced in the early 2000s, for instance, was released first in theaters, which audiences bought tickets to see. About three-four months later, it would be available on home video, for purchase, rental, or pay-per-view. About a year after theatrical release, it would go into the “pay 1” window and be available on a premium service, like HBO, for which consumers paid a monthly subscription, and then on to the “free” TV window, then “pay 2.” The details don’t matter. The point is that was very different from TV or music or anything else.

• Regulatory regimes, rights structure, legal frameworks. Different regulations sprung up around all these different media, sometimes governed by different regulatory bodies. Rights structures, too. I dare you to spend 1/2 hour trying to understand the byzantine rights structures in music, who pays whom and for what. Performance rights, publishing rights, mechanical rights, sync rights, Harry Fox, PROs. Your head will explode.

• Support structures, institutions, even nomenclature. Agencies, consultants, lawyers, accountants, accreditation bodies, unions and guilds, trade press, and awards ceremonies may all specialize in just one or two forms of media. Each medium also developed its own specialized vocabulary, even for very similar things. Just one example, among thousands: a director (film), showrunner (TV), creative director (games), editor (news), and producer (music) are all basically the same thing: the person with primary creative responsibility.

In many cases, these differences were what you would call path dependent: arbitrary outcomes, a function of timing or historical accidents. Also, they are increasingly meaningless to consumers. Consider the differences between, say, movies and scripted TV, which have historically been treated as different businesses. Both tell a fictional story using video. In recent years, TV series have become more cinematic and “movie-like” in terms of production values, budgets, and talent. Throw in “limited series” on TV (those four-six episode, one-season series on Netflix) and the distinction between them looks even blurrier and more arbitrary.

This lack of universality also forced distribution to be physical, local, and capacity-constrained and, as a result, imposed high marginal costs. Movies required reels and screen capacity. TV required spectrum allocation and cable headends. Books, magazines, newspapers, and music needed printing/pressing plants, trucks, and shelf space. The high cost and inflexibility of physical distribution constrained the entire business: the kinds of content that could be produced, the workflows used to create it, where and when it could be distributed, how it could be licensed, and how it was priced and monetized.

Analog media were continuous in the physical sense, but discontinuous in every functional sense.

Era 2 – Math Breaks Distribution (Media as Representative Symbols)

Then came digitization, the mathematization of media. It was a monumental shift. It changed the substrate to discrete and began the process of making the business continuous.

Figure 1. Increasing Sampling Rates

Source: Izotope (https://www.izotope.com/en/learn/digital-audio-basics-sample-rate-and-bit-depth.html).

Also from Infinite Content, Chapter 3:

Digitization involves sampling a continuous sound wave or image at regular intervals, “quantizing” it and converting that quantity to binary code, comprising bits. As shown in Figure [1], the idea is that the more frequent the sampling, the closer it approximates the original signal…

…To hammer home the key point, it is now possible to encode all information—sounds, images (moving or not) and text—the same way. So, while in an analog world, there was no common language among media, digitization changed that. It made the atomic unit of all media the same: the bit.

So, this quantizing, or mathematizing, of media converted it into symbols. All media now comprises some combination of 0s and 1s.

Symbols have many important properties, among them:

• If a domain is mapped to symbols, the symbols can represent anything in that domain—they have universality within that domain

• They can be copied

• They can be processed

• They can be transmitted

Symbols can have other critical properties too, but we’ll get to that.

So, once converted to symbols, all media could now be stored, copied, processed, and transmitted the same way. This, of course, paved the way for “the internet,” which was not just a network of networks, but a suite of technologies (TCP/IP, wireline and wireless broadband infrastructure, compression algorithms, modulation schemes, content delivery networks, software-defined networking, etc.) that made it possible to cheaply and reliably distribute media anywhere.

The migration of media online meant that it was no longer constrained by the cost and inflexibility of physical infrastructure. Marginal costs plummeted. That began the process of breaking down the distribution boundaries that had formed in the analog world:

• Distribution infrastructure. Most media is now transmitted and consumed on just one network, the internet.

• Form factor. Likewise, most media is also consumed on just a few form factors, like mobile, PC/desktop, and connected devices.

• Geography. Media is no longer bound by physical geography, but instead by local regulations and market structures.

• Access. Distribution used to be constrained by time: TV schedules, movie showtimes, newspaper deliveries, and retail hours. Now, everything is constantly available.

• Formats. A movie was a movie because it was in a theater; a newspaper article was defined by being in the newspaper; a TV show existed because it was on TV, etc. Now, formats are mostly defined by modality and use case: Is it video, image, or audio or what combination of those? What sort of “job” does it do for the consumer? Social? Emotional? Functional? A David Fincher project is the same, whether it is called a “movie” or a “show.” A news article is the same whether produced by The New York Times (a newspaper), Vox (a digital news outlet), CNN (a news network), or Jim Acosta (on Substack).

• Economic models. Each format used to have its own relatively rigid way of monetizing. Now, any type of media can sustain many models: advertising, subscription, purchase, donation, microtransaction, bundles, affiliate commerce, etc.

• Rights. Rights used to be based on geography and form of physical distribution. For instance, a film studio would negotiate theatrical distribution with screen exhibitors, retail distribution with big box retailers and rental chains, pay distribution with premium networks, and TV distribution with cable and TV networks—with separate negotiations for each of these channels in each major market. Today, it is far simpler. Netflix seeks global rights across all windows.

There is tremendous economic, institutional, and cultural inertia preserving these old definitions, as I wrote about in A False Sense of Stability. (After all, AOL just recently shut down its dial-up internet service more than 20 years after the commercialization of consumer broadband.) There is also plenty of resistance from anyone who benefits from these historical separations. But the arc of technology is clear. These distinctions are largely historical artifacts and increasingly irrelevant. Much of the consternation and displacement in the industry today is happening because some people are losing ground as these silos slowly break down.

Much of the consternation and displacement in the media industry today is happening because the silos are all slowly breaking down.

Even though the historical boundaries around distribution are crumbling, today the barriers around creation are mostly intact. In the internet age, who creates and how they create has evolved as entry barriers have fallen and tools have gotten better and cheaper—that’s why the creator economy exists. But creation is still time consuming and difficult. And other boundaries, such as between modalities, the distinction between creator and consumer, and the distinction between creation and distribution are firmly in place.

Now those are set to fall, too.

Era 3 – Math Breaks Creation (Media as Combinatorial Symbols)

Generative AI (GenAI) is not an isolated development. It is the logical continuation, and probably culmination, of a forty-year process: turning media into symbols that can be stored, manipulated, and recombined.

To understand how profoundly GenAI will change the status quo, it’s helpful to start with how the models function, at a conceptual level. Whether ChatGPT, Midjourney, Suno, or Veo, they all work the same way:

• They mathematize a training set. In an LLM, that means that every discrete token (a word, part of a word, or punctuation) is assigned a discrete ID, like 7, 9, or 53,781. In an image model, that means every pixel in every image is assigned a color value (between 0-255 for each of red, green, and blue, for 16.8 million possible values). It gets more complicated in audio and video datasets, but the idea is the same.

• That training set may contain one or more modalities (text, image, video, and audio).

• During training, models learn the structures, patterns, and relationships within those data, which they also symbolize, in the form of tokens, embeddings, tensors, vectors, and/or weights. In the case of multimodal models, they learn not just the relationships within each modality, but between them too (such as, say, the relationships between text and images). This internal representation is known as the latent space.

• After training, when presented a condition—whether text, images, video, audio, or anything else—they mathematize the input and contextualize it by placing it in the same or a related representational space.

• This conditioning nudges the model in specific directions within the latent space. Based on this nudging, the model probabilistically produces an output.

• This output is converted back to the native modality or modalities, so that humans can understand it.

The internet exploited the ability of symbols to represent anything (within a domain); GenAI exploits the ability of symbols to create anything (within a domain).

So, GenAI models capture the patterns, relationships, and structures within media. They introduce a new symbolic layer: not just a layer of symbolic representation, but a layer of symbolic context and meaning. These symbols have other important properties beyond the ability to represent, copy, process, and transmit anything within a domain. Within that same domain:

• They can be combined.

• They can be transformed.

• They can be composed.

• They can generate entirely new structures.

In other words, once the relationships are symbolic and manipulable, the system can generate entirely novel combinations, not just store old ones.

The key concept to grasp here is the latent space, a high-dimensional, continuous mathematical space that captures the structures and relationships within data, often across modalities.

Figure 2. A Conceptual Three-Dimensional Embedding Space

Source: Weaviate.

Figure 2 shows a simple, three-dimensional representation of the latent space of a hypothetical language model. The representations, or embeddings, reflect the semantic relationships between tokens. “Wolf,” “dog,” and “cat” are close to each other, as are “banana” and “apple.”

Now, imagine that same space with 100,000-200,000 discrete tokens and hundreds or thousands of dimensions. While you’re at it, imagine that there is a shared latent space between multiple modalities. Such a high-dimensional space captures incredibly nuanced and sophisticated learned relationships within data. It is bounded by its domain—its training data and architecture—but the potential combinations within that domain are effectively unbounded. That’s why moving through the latent space it is possible to combine features and generate text, images, audio, and video that have never been created before.

Because everything in a model lives in the same continuous representational space, once we have enough data and compute, it will be possible to change, remix, or regenerate any media element at any time, cheaply and reversibly. That will make creation continuous and fluid. Matthieu Lorrain, Creative Lead at Google DeepMind, who posts frequently on LinkedIn about the latest GenAI developments (and is worth a follow), often ends his posts with this line: “Content is Liquid.” That’s what he means.

Just as many of these historical distinctions that existed before the internet are still hanging on today, none of this will happen overnight. And as I have written before, there are still important unresolved questions about GenAI, including the degree of consumer acceptance (and for which use cases) and legal issues.

Nevertheless, over time this shift will collapse the remaining, vestigial boundaries in media:

• Workflows and labor roles. Obviously, models that can synthetically create media from prompts will eliminate the need for a lot of labor. They will also flatten workflows and labor roles. Because they are general models, they capture multiple aspects of the production process that have historically been highly specialized. In film, this includes story structures, dialog, character development, cinematography, lighting, editing, sound design, etc. In music, it includes composition, arrangement, instrumentation, performance, mixing, and mastering. In games, it includes worldbuilding, character design, animation, environment art, physics/engines, level design, dialog trees, and rule systems. In print and text, it includes research, outlining, drafting, rewriting, copy editing, fact checking, and even layout. In each medium, the point is the same: roles that evolved around the limits of legacy workflows collapse into less granular levels of specialization or maybe even a singular creative capability. It will enable small teams or even one person to do much more.

• Stages of production. Traditionally, the stages of production in media were precise and the order mattered. Each stage transformed the work into a more hardened state and transaction costs were very high to go back, because the ripple effects of a change would almost certainly affect every subsequent stage. If all media is a continuous latent space, then changes at any point are cheap and reversible and the effects of those changes propagate automatically. The stages collapse into essentially one stage—continuous iteration.

• Creator vs. consumer. There has also been a very clear distinction historically between the people who make and the people who consume, for the obvious reasons. Making is expensive and usually requires time, specialized skills, scarce tools, and institutional support. Once the models near real-time generation, costs of creation fall, and the tools become more user friendly, none of this will be true. Consumers will increasingly be able to remix, reimagine, and perhaps even co-create content.

• Creation vs. distribution. The internet lowered the costs of distribution, which gave rise to a whole new creator class. But today there is a still a clear distinction between creation and distribution. A video, show, movie, book, article, game, or song, is created, “finished” or “locked,” and then distributed. With GenAI, however, the content is never finished because it can be continuously refined. Eventually, it will also be possible to generate it when delivered, such as for personalized, contextual, or interactive content. At that point, the distinction between making and distributing will also get blurred.

When all modalities are mapped onto a shared space, formats stop being containers and instead become different expressions of the same idea.

• Modalities and formats. In multimodal models, different modalities are mapped into a shared latent space. So, the lines between modalities starts to blur, too. A full-length basketball game could be automatically cut into social clips; it could be transformed into an article; or even inspire a song. A news report could become a podcast; a podcast could become an animated explainer; that explainer could become an interactive infographic. (There will also probably be new formats that exist in the spaces between modalities for which we don’t currently have names—liminal formats.) Formats stop being rigid containers and instead become expressions of the same underlying concepts. Once every modality lives in the same representational space, “video,” “audio,” “text,” “game,” and “image” no longer describe different kinds of media so much as different outputs.

  As format, state, and provenance become fluid, pricing models must too.

• Economic models. As I described above, the internet unshackled media from physical infrastructure and its associated high marginal costs, making all media capable of supporting many economic models. GenAI takes this a big step further. Now, a single underlying work can generate effectively infinite versions, infinite lengths, and infinite formats. This will probably have profound implications for economic models, in ways we can’t foresee. When nothing is fixed, pricing probably can’t be anchored to fixed units either. What does this mean exactly? Hard to say. Economic models probably need to shift from selling a unit of something (a movie, an episode, a bundle) to selling access, context, or outcomes. As format, state, and provenance become fluid, the models must too.

Not That Everything Will Change, But Everything Can Change

This brings us back to where I started. GenAI is not an isolated innovation. It is the natural progression of the mathematization of media. The internet capitalized on the ability of symbols to represent media. GenAI capitalizes on the ability of symbols to capture context and meaning within media and recombine to create genuinely novel content, bounded only by the domain.

Over time, that is poised to change the underlying logic of media in almost every way: how it is defined, developed, created, distributed, marketed, funded, and monetized. Given the scope of this change, it is understandably hard for people to get their heads around it, especially those who have been in and around the media business for a long time. Like I said, the more immersed, the deeper the mental grooves.

Here’s an example of what I mean.

When I talk to people about the effects of GenAI in Hollywood, they mostly think about it as a cheaper way to make TV shows or movies. That is a superficial (and skeuomorphic) way to approach it. The Hollywood studios, who are, in general, the most stuck in their ways, are experimenting with plugging AI tools into their existing workflows—like using it for concept art, previs, “subbing and dubbing,” or VFX. GenAI-first studios shake their heads at this. They don’t have existing workflows, they are designing entirely new workflows and new job descriptions, built around GenAI. As a result, they are contemplating much larger savings in labor, time, and cost than the major studios. But I believe that even many of them are not thinking expansively enough. If you dig a little beneath the surface, you can see that their business models are quite similar to traditional studios—similar development processes and funding and monetization models. The point of this example: even those positioning themselves at the cutting edge are arguably not thinking broadly enough about how much things might change.

At this point, some of you may be scratching your heads and asking whether this isn’t all a bit dramatic. Won’t people still want to come home, turn on the TV, and watch a movie together? Won’t they still be interested in whatever is the contemporary equivalent of Taylor Swift’s latest album, GTA VI, or the last season of Stranger Things? Of course they will. I am not arguing that the way people create or consume media has to change in every case; I am only arguing that it can. In other words, creation and consumption will no longer be constrained by formats, modalities, state, or provenance. They will only be constrained by the creator’s intentions and the consumer’s objectives.

To hammer home how big a change this is, think about it this way. Marshall McLuhan wrote that “the medium is the message” more than 60 years ago. What happens when the medium itself gets abstracted away? Is the message the message?

Is the medium still the message when the medium itself gets abstracted away?

Another point is that all of this will, of course, take time. The changes I’m describing will certainly be measured in years and, in some case, probably decades. As I wrote earlier, there is tremendous economic, institutional, and cultural inertia preserving these old distinctions. I oversimplified above and mentioned that most media is now consumed on the internet, but there is also plenty of time spent on traditional forms of distribution—in the U.S., about half of TV time is still on broadcast TV and cable, people still read physical newspapers, three-quarters of a billion movie tickets are sold per year, one-third of audio time is spent with radio. When you launch Netflix, the UI still offers you categories like “Award-Winning Movies” and “Inspiring TV Shows.” Regulators still treat similar media very differently. And so on. The external structures around these traditional forms can hang on for a long time.

My main message, however, is that the arc of technology is clear. It’s worth thinking about the implications now.

Boundaries House Value Pools, So Value Redistributes When They Fall

The boundaries that define businesses also contain their associated value pools. When the boundaries fall, value redistributes.

The big question is, of course, how will value redistribute in this emerging world order? That’s a topic for next time.

1. If we date the beginning of mass media at the invention of the Gutenberg printing press.

2. Radio requires less bandwidth and uses lower frequencies to enable long ranges; TV requires a lot more bandwidth and tends to trade off higher fidelity for a shorter range.