🌻 we know more than we can tell

july links & scraps

I’ve been imagining my life as like steering a big ship. The ocean is wide and deep and looks the same from 360 degrees of vantage. I know there is land, a port, a destination somewhere out there, but after months at sea, it is hard to believe that it exists. Most days you’re just sailing into the big blue. Sometimes you realize you’ve looked at the map or stars wrong and you’re actually off track, and you go oh fuck and start pulling the wheel hard-left, and the rudder is presumably turning a few decks underneath, but you can’t see it so the boat seems to be moving achingly slow. It’s satisfying to realize the misdirection, frustrating to actually see it through. You’re not a nimble kayak that can make hairpin turns around rapids and rocks, not anymore. There are baggage and commitments to carry; things that seemed like gifts but have now become burdens. Weeks between deciding to change course and making it onto the new one. But it’s an accomplishment to pivot nevertheless. In three weeks, everything will be perfect, I tell myself.

---

My theory is that people watch rom-coms to have a vicarious experience of the perfect conversation. Our ordinary dating lives are riddled with shaky small talk, awkward silences, jokes that don’t land, misunderstandings that never resolve. You never come up with the perfect quip until the morning-after shower or the cab ride home.

The rom-com, then, is a form of wish fulfillment. Its goal isn’t realism, though you need enough build-up to make the love story believable. The banter is witty and flirty and never too mean, each peccadillo—a kiss! a spill!—an opportunity to bring the leads closer in their human faults. Conversations are orchestrated like partner dances—each person dipping and spinning and synchronizing, waltzing toward resolution by the 120-minute mark. Actors perform to charm the audience as much as each other; we all wish we could be so clever. Cliches are quickly forgiven if executed well. If only real-life courtship had such excellent rhythm!

Too bad they don’t make good rom-coms anymore. I assumed Celine Song’s Materialists would be as inoffensively cute as Past Lives, but it was even emptier than I expected. The cinematography was picturesque—each still of Pedro Pascal’s apartment looked straight out of an Italian design catalog—but Song’s script eschewed sparkling riffs to make each character a ham-fisted mouthpiece for a scripted theme. DATING IS MORE THAN CHECKING BOXES! MARRIAGE IS AN ECONOMIC TRANSACTION! they announce. Supposedly movies are dumber now because streaming platforms expect viewers to be on their phones and thus inattentive to subtext; for Materialists, my friend and I tried to predict the plot based on the IMDB description and got everything right. Bring back mystery, please.

The other half of Song’s error was trying to make more than “just a rom-com.” She wanted to be above-it-all, a 21st century woman, too meta-aware of gender and class politics to direct a simple love story. And yeah, sure, Materialists is accurate to the Peak Hinge Era we live in, but I don’t want big-screen immersion into modern malaise—I go to the movies to escape it.

On the other hand, seeing When Harry Met Sally at the Roxie was a total delight. I was grinning for probably 80% of it.

---

My favorite piece of AI journalism I’ve read lately is Gideon Lewis-Kraus’s 2016 profile of language modeling at Google Brain. It blows my mind that this was published nearly 10 years ago.

His explanations of technical concepts sit in this perfect trifecta of amusing, accurate, and accessible—I want all AI papers summarized like this. I’ll quote at length here, but the full 15,000 word read is worth it.

On the brittleness of symbolic AI:

Imagine you want to program a cat-recognizer on the old symbolic-A.I. model. You stay up for days preloading the machine with an exhaustive, explicit definition of “cat.” You tell it that a cat has four legs and pointy ears and whiskers and a tail, and so on. All this information is stored in a special place in memory called Cat. Now you show it a picture. First, the machine has to separate out the various distinct elements of the image. Then it has to take these elements and apply the rules stored in its memory. If(legs=4) and if(ears=pointy) and if(whiskers=yes) and if(tail=yes) and if(expression=supercilious), then(cat=yes). But what if you showed this cat-recognizer a Scottish Fold, a heart-rending breed with a prized genetic defect that leads to droopy doubled-over ears? Our symbolic A.I. gets to (ears=pointy) and shakes its head solemnly, “Not cat.” It is hyperliteral, or “brittle.” Even the thickest toddler shows much greater inferential acuity.

Breakthroughs in unsupervised learning:

What the cat paper demonstrated was that a neural network with more than a billion “synaptic” connections — a hundred times larger than any publicized neural network to that point, yet still many orders of magnitude smaller than our brains — could observe raw, unlabeled data and pick out for itself a high-order human concept. The Brain researchers had shown the network millions of still frames from YouTube videos, and out of the welter of the pure sensorium the network had isolated a stable pattern any toddler or chipmunk would recognize without a moment’s hesitation as the face of a cat. The machine had not been programmed with the foreknowledge of a cat; it reached directly into the world and seized the idea for itself. (The researchers discovered this with the neural-network equivalent of something like an M.R.I., which showed them that a ghostly cat face caused the artificial neurons to “vote” with the greatest collective enthusiasm.)

The magic of embeddings:

You want, for example, “cat” to be in the rough vicinity of “dog,” but you also want “cat” to be near “tail” and near “supercilious” and near “meme,” because you want to try to capture all of the different relationships — both strong and weak — that the word “cat” has to other words. It can be related to all these other words simultaneously only if it is related to each of them in a different dimension. You can’t easily make a 160,000-dimensional map, but it turns out you can represent a language pretty well in a mere thousand or so dimensions — in other words, a universe in which each word is designated by a list of a thousand numbers.

And the endearing characters who make up AI research:

Le gave me a good-natured hard time for my continual requests for a mental picture of these maps. “Gideon,” he would say, with the blunt regular demurral of Bartleby, “I do not generally like trying to visualize thousand-dimensional vectors in three-dimensional space.”

The piece is also just insanely prescient:

And in the more distant, speculative future, machine translation was perhaps the first step toward a general computational facility with human language. This would represent a major inflection point — perhaps the major inflection point — in the development of something that felt like true artificial intelligence.

Remember, the general capability of LLMs was extremely non-obvious in 2016. At the time this was published, OpenAI was still spending their Elonbucks doing RL on Dota!

---

I’m not a good book-reader these days, to be honest; I am reading the first 20% of many more books and rarely completing any.

One I did finish is Dan Wang’s Breakneck: China’s Quest to Engineer the Future, forthcoming on August 26 (pre-order it here!). If you’ve read the China letters, you know what to expect. The book centers on the twin foils of China’s “engineering state” versus the US’s “lawyerly society.” In six case studies, he illustrates how the same hard-headed operational excellence that makes the Chinese great bridge-builders makes them dangerous social engineers:

Does the country have too many people? Beijing's solution was to prohibit families from birthing more than one child through mass sterilization and abortion campaigns, as the central government ordered in 1980. Is the novel coronavirus spreading too quickly? Build new hospitals at breathtaking speed, yes, but also confine people to their homes, as Wuhan, Xi'an, and Shanghai did to millions of people over weeks. There is no confusion about the purpose of zero-Covid or the one-child policy: the number is right there in the name. [...]

Officials brought a literal-mindedness to enforcing zero-Covid that created situations best described as whimsical. The coastal city of Xiamen swabbed the mouths of fresh-caught fish to test for Covid. A panda research base in Chengdu tested every animal in its facility. Medical workers chased down Tibetan and Mongolian herdsmen who probably saw nothing but yaks for days on grassland steppes to swab their mouths.

This passage also displays Dan’s deftness for bringing humor into some pretty dark places. From my Goodreads review:

Wang sits in China’s complexities without watering down either side (which many Western commentators struggle with). The achievements coexist with the atrocities, emerging from the same strange place. And Wang's judgments are dealt from a place of both analytical distance and first-person love.

Breakneck is also worth reading because it’s fun: it’s a political economy book with chutzpah, as China letter aficionados will expect. Within two pages the US and China get called “full of hustlers” and “utterly deranged,” while Europe is a “mausoleum economy… too sniffy to embrace American practices.” Somehow he got away with calling America "low-agency" and "low-T."

Wang’s voice shines through the text, which is, I think, the best compliment a writer can get.

I’ll do a podcast with Dan when the book is officially out in late August (and after I return from my own 2.5-week trip to China). Tune in:

---

Another running theme in Breakneck is “process knowledge,” also similarly described as “mētis” by James C. Scott or “tacit knowledge” by the scientist-turned-philosopher Michael Polanyi (“we can know more than we can tell”).

While some information can be codified in blueprints and guidebooks, other skills are embodied, intuitive, and learned via the act of doing. In cooking, it’s knowing how to hydrate bread dough and season a stir-fry through feel, not measurement. In Scott’s Seeing Like A State, he contrasts the failed monocrops of scientific forestry to peasant farmers who adjust to local climate cues. In Breakneck, it’s the “living practice” of engineering—the educational institutions, the dense geographic networks, the way engineering shows up as a cultural value—that makes Shenzhen a manufacturing powerhouse while the Upper West Side is not.

Tacit knowledge is a common counterargument against fast automation timelines. The more critical knowledge you believe is not written in any book or site, the harder it’ll be for models to learn. It’s the gap between reading a recipe and cooking a meal, between pick-up lines and face-to-face chemistry, between The Lean Startup and having real product taste, between “seeing” the scaling laws and “feeling” them in your bones. Tacit knowledge is the reason that other people’s work looks trivial to automate, while your own job always seems infinitely complex.

But I actually think tacit knowledge is a big reason for the unreasonable effectiveness of deep learning. The big-model approach can glean patterns and concepts from massive unlabeled datasets; as it turns out, there’s tons of tacit, contextual knowledge embedded in web text—certainly more than most of us could’ve predicted in 2015, more than any symbolic AI approach managed before. Neural networks can infer fuzzy concepts like “charisma” from rom-com scripts, even if they’re never spelled into rules. “We know a person's face, and can recognize it among a thousand, indeed among a million. Yet we usually cannot tell how we recognize a face we know,” continued Polanyi in The Tacit Dimension. During the 1950s cybernetics craze, he challenged: How could machines recreate the human mind if they still needed humans to define the rules? Polanyi should’ve seen Quoc’s cat paper.

---

Ellen Ullman’s Close to the Machine is a memoir I couldn’t believe I hadn’t read earlier. Many of her observations about engineering culture feel as relevant today as when she was a programmer in the 1980s: the vast communication gap between makers and managers, or the way real-world applications fade into logical abstractions when in the thicket of a bug:

Guide an X-ray machine or target a missile; print a budget or a dossier; run a city subway or a disk-drive read/write arm: it all begins to blur. The system has crossed the membrane—the great filter of logic, instruction by instruction—where it has been cleansed of its linkages to actual human life.

Also, she may be the one person before Dan to discuss the “lawyerly” versus the “engineering” state:

When the Soviet Union began to crumble, and the newspapers wrote about the men who controlled the empire, I couldn’t help noticing how many of them had been trained as engineers. Our country is ruled by lawyers, I thought, theirs by engineers. Engineers. Of course. If socialism must be “constructed” (as we said in the party), if history is a force as irrefutable as gravity, if a “new man” must be built over generations, if the machine of state must be smashed and replaced with a better one, who better to do the job than an engineer?

The book’s title is also just extremely good; much of Reboot’s new mission was inspired by thinking about how tech writing can be “close to the machine.” Recommend.

---

In India, there may be an even bigger gap between records and reality, as

Arjun Ramani

writes in his essay on spending a year on the ground as The Economist’s India correspondent. My favorite bits were his first-person reporting anecdotes:

In the West, it is quite possible to read some bank analyst reports, crunch some data, scroll Twitter, talk to a few experts and write about economics for a popular current affairs magazine. This is not to diminish the job, just that it doesn’t necessarily require much shoe-leather reporting. By contrast, reporting in India will burn through pairs rather quickly.

India’s underground economy is vibrant. Despite demonetisation, I’m confident a big chunk of political donations and property transactions still happen in cash (I asked a lot of developers and campaign operatives). Corruption and crime are commonplace–more than 40% of Members of Parliament have an ongoing criminal investigation! Business and politics happen over WhatsApp and telephone calls. India’s Internal Tech Emails would be a bore. A good chunk of red tape from the Licence Raj is still on the books, making rent-seeking a necessity for business. Amazingly, prohibition is still active in Bombay so you need a liquor licence to drink, but basically no one has one! Economically, the majority of activity is “informal”, or in small unregistered firms that do not pay tax. As a result, many economic statistics are best described as educated guesswork. [...]

I was doing a background interview with a well-known politician running for election. In the middle of the conversation he took a phone call. I still don’t understand why he didn’t go to another room. Maybe he assumed I didn’t know any Hindi. In any case, the politician was asking a businessman for black money as he had run out of campaign resources. “Liquid chahiye” (I need cash) was the tell. After the call, he looked at me sheepishly and explained that there was no other way to survive without cash. You need at least 6-7 crore rupees ($1m) to run an MP campaign, he said (eight times the legal limit). Gesturing to his bungalow, he protested: “it’s not for me, I don’t even live in a big house!”

Delightful and informative all around. I will never tire of the “Notes on [place]” format.

---

I’m spending half of August in China: a week in Shanghai, and a few days each in Hong Kong, Shenzhen, and Hangzhou. A friend asked if I was going for family reasons or “writing/tech/abundance reasons,” which I found quite amusing because the answer is exactly 50% of each.

I learned last year that I’m a much more attentive traveler when I’m not-so-secretly hoping to get a post out of the experience after—which is a little cringe, but also useful enough that I’m still going to do it. So expect a bit of quiet in mid-August, and then what I hope will be an essay and podcast about the trip.

If you have recommendations for non-obvious things to do or people to meet in those places—particularly food, art, and tech—or place-relevant books/films to consume on the plane—do comment or email! I’ve been to Shanghai and Hong Kong before, but it’ll be my first time in Shenzhen and Hangzhou.

---

Nadia

Asparahouva’s Antimemetics put a word to a concept I’ve been circling for a while. Not the titular antimeme, but its converse.

A “supermeme” is an idea that is characterized by its “gravitational pull,” the way it sucks up attention due to characteristics like an appeal to strongly held values, perceived widespread impact, and lack of specificity (e.g. AI, climate, nuclear war). She writes:

Supermemes often take the form of a civilizational threat that demands us to prioritize it above everything else. In the vein of “No one ever got fired for buying IBM,” doomsday scenarios are easy to justify working on, because “No one can blame me for wanting to save the world from destruction.”

The reason I've long been suspicious of x-risk discourse is that it’s often used as a conversational trump card, crowding out everything else with sheer scale and moral weight—because what could be worse than extinction? Like an invasive species, supermemes have traits that make them extremely fit for winning in abstract attentional marketplaces. They're rhetorically powerful and/yet unfalsifiable.

(I first noticed that the same harms that won high school policy debate rounds tended to be the ones that drew cult followings in late 2010s Twitter. Both environments reward rhetorical fearmongering with no care for action or implementation.)

---

My favorite podcasts are the ones where we’re not just summarizing each other’s work, but making new breakthroughs in the live process of conversation.

In my (very fun) conversation with Fred Turner, we happened upon the notion of the “get my bag” economy—a particular Gen Z nihilism characterized by the idea that every system (college, job interviews, dating, politics) is a scam, so you might as well grift your way to the top and win. Meekness and morality are equally scorned; winners are venerated no matter how they got there. I think it’s impossible to disentangle this phenomenon from the fact that Donald Trump has been president for zoomers’ entire adult lives. We now live in the Cluely age.

So when I began

Francis Fukuyama

’s End of History and The Last Man, I was attuned to his meditations on generational nihilism—in this case, resulting from the destruction of the world wars:

While Stalinism did arise in a backward, semi-European country known for its despotic government, the Holocaust emerged in a country with the most advanced industrial economy and one of the most cultured and well-educated populations in Europe. If such events could happen in Germany, why then could they not happen in any other advanced country? And if economic development, education, and culture were not a guarantee against a phenomenon like nazism, what was the point of historical progress?

The experience of the twentieth century made highly problematic the claims of progress on the basis of science and technology. For the ability of technology to better human life is critically dependent on a parallel moral progress in man.

It’s hard enough to steer an individual or a company back from folly. What about an entire nation?

---

I’m getting into screen-time blocking. I use the app Opal on a recommendation from

Lucas Gelfond

. It won’t let me scroll Twitter, Substack, or Instagram for 80% of the day. Gmail is also blocked from 11pm to 8am. The app feels like the correct amount of restriction: the torturous “Take a break” screen is just inconvenient enough to allow appropriate uses but block the unnecessary ones, and my average phone use has dropped by about 30-60 minutes each day.

---

Also, spaced repetition. I initially started making Anki (well, Mochi) cards to keep track of Google DeepMind’s LLM lineage—Gopher, Chinchilla, Gato, PaLM, Sparrow, Meena, LaMDA, Bard, Gemini, etc. Then I started adding new words, papers, philosophy concepts, and Chinese slang. Anything and everything gets thrown into the deck.

Turns out the tools for thought guys are right: regularly reviewing flashcards is vastly more effective than a one-time lookup. I feel a hit of accomplishment whenever I then hear my learned words in the wild (“genuflect” while seeing Giant at West End) or manage to sneak one into a post (“bien-pensants” in my Breakneck review, “peccadillo” in this one). And given the whole phone-screen feed-blocking thing, I’m trying to review my flashcards instead whenever I get too fidgety.

PS: The bottom-right corner is the most important part of your phone home screen, because it’s the easiest for your thumb to reach. Whatever sits there becomes instinct. Pick carefully!

---

Which way is the ship steering? It’s been over 6 months since I became an “independent writer”—a half-year spent dipping my toes into every form of media production I could, from freelance reporting to book-writing to anonymous interviews to consulting projects to pay the bills. I’m also playing with a speculative fiction piece that may or may not get good enough to share.

After all that experimentation, I’m still the proudest of my more idea-driven Substack essays—in part because they are also the hardest to write, and thus most satisfying to land. (If you didn’t suffer, did you really learn?) But essays require much longer incubation periods for research and spawning surprising connections, so they’re the first to get sacrificed when I’m spread too thin.

I’m refactoring my daily schedule to maximize production of those bigger essays. More open calendar blocks to till the soil, more exploratory reading to fertilize the seeds of thought. More opportunities to nurture rough drafts into maturity, in the hopes that at least one will be ready for harvest each month. There will be other things too, e.g. maybe more link roundups like this one—do tell me what you’d like to see more of!

Thank you for following along with the experimentation. In three weeks, everything will be perfect.

---

Tomorrow is my 26th birthday. If you’d like to get me a gift, consider a paid subscription :) I’m only 4 away from hitting 100, and it means a lot to have reader support while I prioritize my independent writing. I’d really like to be able to continue doing this next year.

(And if you aren’t able to, consider telling a friend about this newsletter / sharing a favorite post! It’d be just as appreciated.)

All the best,

Jasmine Sun