Xark!

2012: Google gives birth to the bottom-up Semantic Economy

Back in 2009, while contracted to work on a doomed content-repository project, a flash of insight struck me: The problem with grand visions of the Semantic Web was that they all assumed a top-down structure. One wickedly clever set of rules to wrangle every fact. A global ontology.

It didn't make sense. Global ontologies are like Soviet Central Planning. Rules are meant to be broken. And top-down systems are crashing and burning everywhere you look.

Plus there was another un-fixable problem: Everyone with money to spend on these projects wanted machines to do the yeoman work. Because machines are cheap.

Think about that for just a moment. We're talking about organizing the sum total and nuance of human knowledge, but the entire world assumes somehow that this is a job for machines. That the best way to understand the complex, pattern-based output of human intelligence and language is to assign computers to decode it after the fact.

People think that makes sense because they think computers are magic, not machines. Meanwhile, in the real world, Text Mining Engines aspire to 75 percent accuracy. That's why our content-repository project failed. The client's product specifications couldn't be satisfied via the vendor's pathetic 75 percent accuracy rate.

So one night I asked myself: Could you reach the goal if you flipped the script on every core assumption? Not top-down, but bottom-up? Not machine intelligence, but human intelligence, assisted by machines? Not one "global graph" but many interconnected "directories of meaning" based on capturing machine-readable statements of fact during the production of human-readable articles?

And of course, the answer is yes, you can do all these things, and you can do them profitably, so long as you follow two simple rules: 1. Build tools that make it easy to publish directories of meaning; and 2. Give users the power to make their directories cooperate with other directories.

Once you do that, the need to create perfect top-down rules for knowledge disappears, because you'll have harnessed the power of emergent properties. If you build a good directory, others will want to use it.

What's so hard about that?

But people didn't get it. Most still don't, for lots of reasons -- including our very human inability to hear anything new without forcing it to fit into old assumptions.

They're about to start getting it, though, because now Google gets it.

The search giant has constructed a bottom-up directory of meaning. The company calls the product "the knowledge graph" and the service Semantic Search. Being Google, the company still sees the problem as a data-recovery challenge, but that doesn't matter. Once such directories exist, a semantic economy based on the value of machine-readable definitions is born. Once we begin feeding that market, information becomes a public commodity. And once we give people the power to define the meaning of their own words, and then to share those meanings in a mutually beneficial way, we'll have tapped into the same emergent property that generated Wikipedia.

It's not that complex. It doesn't require any exotic programs. But it does take vision, discipline and the tiny bit courage required to buck the status quo.

My essays on the topic (Semantic Journalism topic).

An FAQ for my semantic journalism essay

Since several readers yesterday seem to struggle with some of the big-picture concepts in my essay about standards-based journalism in a semantic economy, I'm taking this opportunity to build a better cognitive bucket for all this information I'm shoveling your way. And it seems like the big problem is understanding what RDF is and how it might work in this context.

Rather than doing this in essay form, I'm going to try it as an FAQ.

Imagining the Semantic Economy

Abstract: This 3,000-word essay proposes and describes an economy based on exchanges of intelligently structured data; argues that the path to that economy will not be imposed from above but will emerge like a crystal lattice in a super-saturated solution; explains why a Semantic Content Management System for the publishing industry could be the profitable catalyst for this development; discusses the outline of such a system.

Clay Shirky blew up the grand vision of The Semantic Web in 2003, which is probably why serious people aren't particularly alert to the possibility of a semantic revolution building in response to current conditions. That's probably because the general perspective on semantic architecture is that it must be grandiose and top-down. I believe the route to our semantic future comes from the opposite direction, and I've tried to make that case by imagining and describing a publishing system that would create information structures that could spread because the publishing system would make those structures profitable.

But the widespread inability to imagine the benefits of such tools points to a more fundamental lack of understanding, and it occurred to me recently that maybe it's time to address it.

It's the Semantic Economy, Stupid.

The Big Pool of Money experiment

At last night's meeting of the Columbia chapter of Social Media Club, I mentioned something I heard suggested in an off-hand way at the Infovalet Conference in May. What if every American paid a fee as part of his or her monthly ISP bill, and then that money were divided up among content creators?

Of course, what I found ironic about this back in May was that the people suggesting it weren't suggesting that ANYONE be able to share in this revenue. The idea was that only content creators who happened to also own printing presses and/or FCC licenses would be eligible. That's absurd on its face, which is probably why that idea hasn't gone anywhere. Try selling that politically.

But as I drove home to Charleston, I reconsidered the idea. Which is why I want you to play along in a thought experiment with me.

What if we all paid into a fund that could be distributed among EVERYONE who creates content, of any kind, that can distributed online? Bloggers, filmmakers, musicians, reporters, pornographers, comedians, ANYONE.

How would it work? How could it be administered? 

The betrayal of the Fourth Estate

I have attempted to be discreet on the decline and fall of the media empire, mostly because I am still in it. With layoffs and furloughs announced every quarter, the consequences of being brutally honest could be dire. But I am disgusted by the complete hypocrisy of the newspaper industry, whose leaders are staking a claim to the sacred art of journalism, as if it has sole rights to it. The assertion by newspaper executives that newspapers are the torch-bearers for a community (or global) moral and ethical center would be laughable, if it were not being used to cloak the greed of corporations that have enjoyed healthy profit margins for decades, all the while failing to invest in anything other than expanding their arrogance. 

These corporations profess it is their duty to hold the world to a higher standard but conveniently overlook their failure to substantively investigate major issues and their own increasing immersion in the very organizations they profess to watchdog. Three words. Yellow cake uranium. Need more? WMDs. Economic meltdown. Climate change. Want a book? Death by Journalism.   Into the Buzzsaw.

(Note to press: You are supposed to COVER politicos not hang out with them.)

From Jefferson to America 4.0

I owe my patriotic sentimentality to my mother, a teacher from Iowa who encouraged me to read and spent the early 1970s prowling yard sales for age-appropriate books about American history, which I promptly consumed. It is because of her that I worshiped Thomas Jefferson above all the other founding fathers, carrying his ideal of intellectual egalitarianism well into my adult life.

Last week she sent me an email titled "Thomas Jefferson's sayings in light of today." My hackles rose before I even opened it, since poor red-haired TJ has become the Patron Saint of Liberty in modern America, cited by all sides as proof of their fundamental wisdom and Americanism. And Mom joined me in ambivalence: "Don't know how I feel about a lot of these ... but it is interesting!"

Interesting and deeply bothersome is how I'd put it.

"When we get piled upon one another in large cities, as in Europe, we shall become as corrupt as Europe."
 
"The democracy will cease to exist when you take away from those who are willing to work, and give to those who are not."
 
"It is incumbent on every generation to pay its own debts as it goes. A principle, which if acted on, would save one-half the wars of the world."
 
"I predict future happiness for Americans, if they can prevent the government from wasting the labors of the people under the pretense of taking care of them."

And so on. In short, the collection presents an American icon as Ron Paul with a pony tail, a Libertarian genius standing athwart history and shouting "Stop!" For every anxious American worried about the future of the country under Democratic leadership, this collection reads like Jefferson's j'accuse from beyond the grave.

Which brings me, this Presidents Day, to an uncomfortable duty. I come to argue not only with one of my greatest heroes, but in doing so, to argue for a new understanding of this particular moment in history. Because I believe we are in the midst of the fourth America revolution.

Obama and "consolidation" of power

Matt Stoller wrote an excellent piece last week about Obama's quiet consolidation of party and grassroots power, based largely on the nominee-in-waiting's unprecedented ability to raise money.

I think the Stoller piece marks the beginnings of our noticing the forest instead of the trees: While Obama opponents continue to paint his success as some Svengali-like personality cult, a more grounded view reveals a hyper-competent modern organization that has managed to integrate multiple campaign efforts (fund-raising, various levels of community organizing, mass media, alternative media,  online, intra-party relations, "the ground game" and yes, the candidate himself).

To me, Obama's campaign represents a great selling point for his abilities as an executive, and I have no doubt that political scientists will be exploring its practices and principles as the first major case study in 21st century American politics. But to Democrats with a long-time stake in the party, Obama's success is both welcome (who wants to live in the wilderness forever?) and intimidating.

Why the angst? Here's a commenter on Stoller's follow-up piece:

The anti-authoritarian impulse common in a lot of liberals, and which I share, is definitely visible on both the pro and anti Obama sides.

Re: the movement, from glancing at other blog comment threads, I'm reminded of a very gut-level, "It's happening without ME, I am NOT included....therefore they will screw ME...."they" have decided I'm worthless...I don't want anybody having power over ME/My favorite orgs" self-absorbed reaction.   

And this reaction is disguised in a "but what I really care about is the party" pose.  It's really a paranoid fantasy.  It's all fear.

Info R/Evolution, Intertwingularity & Xarkism

I built this blog began in the summer of 2005 based on a couple of insistent thoughts:

1. The standard media/cultural categories for topics and discussions were entirely too sterile and limiting for the way I wanted to think and talk;
2. Based on my mode of working as a reporter (diving intensely into one topic after another) it was increasingly obvious to me that my learning in one area (quantum physics) influenced my thinking about another subject (microbiology), which provided insight into seemingly separate topics (mass media, sociology, politics, etc.).

Hence, Xark began with a foundational statement: Because there are no unrelated topics.

Our thought? Maybe by involving people from multiple backgrounds in multiple topics, we'd have more interesting and productive discussions and insights. I based this on the notion that communites that grow up around "themed" blogs tend to evolve into monocultures. Ecosystem biology teaches us that a monoculture (tree farm) simply isn't as sustainable, healthy or as valuable as a naturally diverse ecosystem (rainforest).

These days I'm happy to observe how well those concepts fit into our developing understanding of knowledge and human intelligence in the networked world. From Peter Morville and his book Ambient Findability to Dave Weinberger and his Everything is Miscellaneous, the leading edge of the culture is rapidly incorporating radical ideas about the semantic structure of information -- quite literally, how the Web works better when we pattern our information systems on human-ness. The Web has rather haphazardly grown into an extension of ourselves. The next step (generically, The Semantic Web) may be very deliberately built as an extension of human consciousness.

So Ted Nelson's notion of Intertwingularity (1974) re-emerges in a new contest and reflects its futuristic light on the notion of Xarking.

Intertwingularity is not generally acknowledged -- people keep pretending they can make things deeply hierarchaical, categorizable and sequential when then they can't. Everything is deeply intertwingled.

So Anthropology professor Michael Wesch begins to make sense instantly: Everything is connected. Nothing is separate.

I suspect it was always this way. Perhaps we saw it differently before because information and communication was so slow and precious and difficult before. It took improvements in maritime and navigational technology before we could "see" the Earth as round. Maybe it takes the explosion of networked media for us to "see" that everything is an expression of the one, that technology is evolution by non-biological means, that political, economic and social systems based on keeping us artificially separate and oppositional are wasteful relics.

The rest of the world doesn't think this way right now. We're still in the minority. But that could change.

Magic = Doing what you imagine

Twenty-six years ago I sold my clothes and books to buy groceries and Bugler tobacco to last me out the final two weeks of school and exams. Twenty-five years ago I had a script for a 10-minute film I could have shot with borrowed equipment if I'd had less than $200. Twenty-four years ago I could have quit my job cutting greens for $4 an hour and made double that if I could have come up with $500 to buy a used pickup truck and a push power mower.

Later that year (1984)  I thought it might be a good move to start a coffee shop in a vacant storefront on Howard Street in Boone, NC, just a few blocks from campus. Thought I could offer people a place to drink coffee and smoke cigarettes and talk talk talk.  I could have pulled it off for less than $1,000 in 1984, and I even had friends who wanted in on the idea. But they were broke too.

And so we didn't start a coffee house, and I joined the Army instead.

I like to remember this now, because it doesn't really seem that long ago that I lived in a world where I could imagine anything but I couldn't do very much about any of it. Not very many people could. The deck was stacked against people without money or access, and we shaped our dreams accordingly.

Another g-damn magnum opus

I just published an enormous overview of my ideas about the future of media ("Foundations of 21st century journalism") over at my revived media blog. I'm rather appalled by how long it is (3,275 words), but it's a big topic. And I'm not writing for everyone.

The subheads:

Monoculture to ecosystem
Structured and semi-structured data
Scalability
Open Source
Informatics
The Blur: News, information and advertising
Newsbots and Intelligent Agents
Multiple revenue streams and business models
The Intelligence Briefing model
Mainstream retrenchment
Nichestreaming
E Pluribus Unum
Watchmen watchers
Credibility grading
Death of monopoly pricing and profits
Game theory
Social technology -- virtual and otherwise
The Web is Local
True Convergence
Curating information
New elites
The Creative Middle Class
Surplus people
Yes, newspapers are going away

This was written for the students and faculty at the Journalism Department at the University of Mississippi, but you're all welcome to have your say.