Epistemic Status: 
Sprouting — I’m ruminating on these ideas, nothing really solid yet, even if I do argue a bit normatively here.
I’ve been thinking a lot about uses for Large Language Models (LLMs) lately. I read a post the other day about using some off the shelf locally run models to build a tool for helping new team members understand a codebase. I think that’s a great use for these things, as long as you take care to consider the tendency of these generative models to bullshit. *The correct term here is “bullshit”. Hallucination implies a perception of the world, which these lack. Bullshiting is a lack of concern for objective truth. See the paper “ChatGPT is bullshit” That’s where your teammates come in, either in conversation about the work, or in code review. Being able to quickly find your way around a codebase without bothering senior teammates with questions could be empowering to junior engineers, and the results they showed look promising.
That said, one concern that I continue to see come up is the one about how these things are trained. Like any neural network (of which LLMs are partially composed) there are weights that need to be set, *Grant Sanderson has a great video series about GPT that explains the details. and the way these are set isn’t by someone manually deciding what these values should be, but by training the network on some training data.
These training data are where the controversy lies. It’s no secret that companies like Meta and Open AI among others have used copyrighted material for training data. There seems to be the idea that somehow these models trained on these data are copying that data in violation of copyright law. It’s easy to feel like these large corporations are exploiting the work of artist, writers, and others, to build these new cash cows.
How different are these models really?
The thing I keep coming back to and asking myself is, while these models aren’t like how our brains work, how different are they really? How far away from what we do is what they do?
Neural networks are supposed to be modeled off the brain, and while they certainly don’t work like the brain exactly — they’re trained to do one particular task — I can’t be sure that what I do when I’m talking or writing or doing anything creative can’t be boiled down to “what’s the most likely next thing here?“. *I don’t subscribe to behaviorism though. Skinner can keep his pigeon boxes. See Punished by Rewards I’m drawing on the context of what came before, what I’ve read before, what I’ve seen before, to come up with what I’m going to do next.
How is it different for me to read a bunch of copyrighted books and then write one of my own that is influenced by those? How is it different when I can recite word for word a passage of a copyrighted song that I’ve heard a thousand times? How is this not the same fair use that I take advantage of when I quote a book for an article? I’m not sure that it is, are you?
I don’t have the answers for these questions. *I have very strong opinions about copyright and intellectual property in general that don’t match with most people’s views. “Nothing is yours, it’s to use, it’s to share.” I just think the situation is more complicated than any knee-jerk reaction is going to take in. The attempts to try to somehow shield or defend human creative works from becoming training data via technical or other means is a fool’s errand, a war that can’t be won.
I’m also reminded of a quote from The Craftsman by Richard Sennett in the chapter on machines.
He’s summarizing Diderot’s take on machines here.
The enlightened way to use a machine is to judge its powers, fashion its uses, in light of our own limits rather than the machine’s potential. We should not compete against the machine. A machine, like any model, ought to propose rather than command, and humankind should certainly walk away from command to imitate perfection. Against the claim of perfection we can assert our own individuality, which gives distinctive character to the work we do. Modesty and an awareness of our own inadequacies are necessary to achieve character of this sort in our craftsmanship.
Which is quite different from the conclusion Adam Smith reached in The Wealth of Nations which Sennett quotes just a paragraph before.
In a factory “the man whose whole life is spent in performing a few simple operations… generally becomes as stupid and ignorant as it is possible for a human creature to become.”
Sennett’s point here seems to be that we shouldn’t measure ourselves against machines, whether it’s a table saw cutting wood faster and more consistently than a trained woodworker with a hand-saw, or AI creating text or speech for a video game. *I’m using video games as an example because I know a little about making them. If anything, it makes those handcrafted creations more valuable (even if the market doesn’t value it so).
They’re tools to use, not things to compete against. Using the text to speech example in a game, I could record the dialog audio manually, or hire someone to do it, but if I’m on a budget as an indie developer and can’t hire someone, or I don’t feel up to all the work, I could create a model based off some samples of character dialog I’ve created myself, and then use those trained “voices” to do the rest. I’d still need to carefully review that work to make sure it’s up to the standards I’m going for.
What really needs to change
The real problem here isn’t that these models are being used by for-profit corporations to create things that would take away human jobs. I actually think some of the uses of “AI” generated art can have a place in the creative toolbox, and don’t take away work from a human.
The problem is that we as a society don’t value things like the arts as we should. Our society is too focused on productivity and profit to care about things like music or opera or theatre. The focus on profit and competition is in fact anti-innovative. The most creative and innovative work happens when people are left to their own devices to explore their own interests.
The fact that an artist can’t support themselves or their family on their creative works alone, can’t get decent health care for themselves or their family, is the real problem. The fact that people can’t take the risk of starting a business if they have a family member who needs health care, is the real problem. There should be a bottom on how far people can fall, so that they can work on the things they truly want to work on without worry about whether it’s going to have any monetary value.