(2024-09-10) ZviM AI #80 Never Have I Ever

Zvi Mowshowitz: AI #80: Never Have I Ever.

Will AI ever make art? Fully do your coding? Take all the jobs? Kill all the humans?

Most of the time, the question comes down to a general disagreement about AI capabilities

if frontier AI capabilities level off soon, then it is an open question how far we can get that to go in practice.

lot of frustration comes from people implicitly making the claim that general AI capabilities will level off soon, usually without noticing they are doing that.

There are also continuous obvious warning signs of what is to come, that everyone keeps ignoring, but I’m used to that.

Table of Contents

• Introduction.
• Table of Contents.
• Language Models Offer Mundane Utility. Sorry, what was the question?
• Language Models Don’t Offer Mundane Utility. A principal-agent problem?
• Fun With Image Generation. AI supposedly making art, claims AI never will.
• Copyright Confrontation. OpenAI asks for a mix of forgiveness and permission.
• Deepfaketown and Botpocalypse Soon. How to fool the humans.
• They Took Our Jobs. First it came for the unproductive, and the call centers.
• Time of the Season. If no one else is working hard, why should Claude?
• Get Involved. DeepMind frontier safety, Patel thumbnail competition.
• Introducing. Beijing AI Safety and Governance, Daylight Computer, Honeycomb.
• In Other AI News. Bigger context windows, bigger funding rounds.
• Quiet Speculations. I don’t want to live in a world without slack.
• A Matter of Antitrust. DOJ goes after Nvidia.
• The Quest for Sane Regulations. A few SB 1047 support letters.
• The Week in Audio. Dario Amodei, Dwaresh Patel, Anca Dragon.
• Rhetorical Innovation. People feel strongly about safety. They’re against it.
• The Cosmos Institute. Philosophy for the age of AI.
• The Alignment Checklist. What will it take?
• People Are Worried About AI Killing Everyone. Predicting worries doesn’t work.
• Other People Are Not As Worried About AI Killing Everyone. What happened?
• Five Boats and a Helicopter. It’s probably nothing.
• Pick Up the Phone. Chinese students talk about AI, safety and regulation.
• The Lighter Side. Do we have your attention now?

Language Models Offer Mundane Utility

Rohan Paul: Simply adding “Repeat the question before answering it.” somehow make the models answer the trick question correctly

We need a good prompt benchmark. Why are we testing them by hand? After all, this sounds like a job for an AI.

Language Models Don’t Offer Mundane Utility

My hunch is that AIs could totally write novels if you used fine-tuning and then designed the right set of prompts and techniques and iterative loops for writing novels. We don’t do that right now, because waiting for smarter better models is easier and there is no particular demand for AI-written novels.

Fun with Image Generation

I continue to be a mundane utility skeptic for AI video in the near term.

There was a New Yorker piece by Ted Chiang about how ‘AI will never make art.’ This style of claim will never not be absurd wishcasting, if only because a sufficiently advanced AI can do anything at all, which includes make art. You could claim ‘current image models cannot make “real art”’ if you want to, and that depends on your perspective, but it is a distinct question

there’s also the ‘so what if it doesn’t feel or desire?’ question.

The obvious philosophical point is, suppose you meet the Buddha on the road. The Buddha says they feel nothing and desire nothing. Did the Buddha use language?

Copyright Confrontation

OpenAI says it is impossible to train LLMs without using copyrighted content, and points out the common understanding is that what they are doing is not illegal

Deepfaketown and Botpocalypse Soon

We evaluated 3 systems (ELIZA, GPT-3.5 and GPT-4) in a randomized, controlled, and preregistered Turing test. Human participants had a 5 minute conversation with either a human or an AI, and judged whether or not they thought their interlocutor was human. GPT-4 was judged to be a human 54% of the time, outperforming ELIZA (22%) but lagging behind actual humans (67%).

Turns out that you could get into airline cockpits for a long time via a 2005-era SQL injection

Voice actors sue Eleven Labs, accusing them of training on audiobook recordings and cloning their voices

Patrick McKenzie explains that Schwab’s ‘My Voice is My Password’ strategy, while obviously a horrible thing no sane person should ever use going forward given AI, is not such a big security liability in practice

They Took Our Jobs

Call centers in the Philippines grapple with AI. Overall workloads are still going up for now. Some centers are embracing AI and getting a lot more efficient, others are in denial and will be out of business if they don’t snap out of it soon. The number of jobs will clearly plummet, even if frontier AI does not much improve from here – and as usual, no one involved seems to be thinking much about that inevitability. One note is that AI has cut new hire training from 90 days to 30.

In a post mostly about the benefits of free trade, Tyler Cowen says that if AI replaces some people’s jobs, it will replace those who are less productive, rather than others who are vastly more productive. And that it will similarly drive the firms that do not adapt AI out of business, replaced by those who have adapted

Time of the Season

*They said the AIs will never take vacations. Perhaps they were wrong?

xjdr: Some days, nothing you try works*

The post-training performed to make claude himself has put him within an llm basin that is european in many ways

We are very clearly not doing enough A/B testing on how to evoke the correct vibes.

It also includes evoking the right associations and creating the ultimate Goodhart’s Law anti-inductive nightmares.

Get Involved

Introducing

Not AI yet, that feature is coming soon, but in seemingly pro-human tech news, we have The Daylight Computer. It seems to be an iPad or Kindle, designed for reading, with improvements. Tyler Cowen offers a strong endorsement, praising the controls, the feeling of reading on it and how it handles sunlight and glare

On the downside it costs $729 and is sold out until Q1 2025 so all you can do is put down a deposit?

Claude for Enterprise, with a 500k context window, native GitHub integration and enterprise-grade security, features coming to others later this year

In Other AI News

Quiet Speculations

Tyler Cowen speculates on two potential AI worlds, the World With Slack and the World Without Slack. If using AIs is cheap, we can keep messing around with them, be creative, f** around and find out*

What is confusing about this is that it divides on energy costs, but not on AI capabilities to create the art at all

This goes back to Tyler’s view of AI and of intelligence, of the idea that being smarter does not actually accomplish much of anything in general, or something?

Here’s Ajeya Cotra trying to make sense of Timothy Lee’s claims of the implausibility of ‘AI CEOs’ or ‘AI scientists’ or AIs not being in the loop, that we wouldn’t give them the authority. Ajeya notices correctly that this is mostly a dispute over capabilities, not how humans will react to those capabilities

those that leave humans meaningfully in charge will get swept aside

At AI Snake Oil, they claim AI companies have ‘realized their mistakes’ and are ‘pivoting from creating Gods to building products.’ Nice as that sounds, it’s not true

They list five ‘challenges for consumer AI.’

The first is cost. You have to laugh, the same way you laugh when people like Andrew Ng or Yann LeCun warn about potential AI ‘price gouging.’ The price has gone down by a factor of 100 in the last 18 months and you worry about price gouging?

The problem is that AI is intelligence, not a deterministic program, yet we are holding it to deterministic standards. Whereas the other intelligence available, humans, are not reliable at all, outside of at most narrow particular contexts.

Your AI personal assistant will soon be at least as reliable as a human assistant would be. (intelligent software assistant vs virtual assistant)

The third problem they list is privacy, I write as I store my drafts with Substack and essentially all of my data with Google, and even the most privacy conscious have iCloud backups

Fourth we have safety and security

Finally there’s the user interface. In many ways, intuitive voice talk in English is the best possible user interface. In others it is terrible.

When you try to use an Alexa or Siri, if you are wise, you end up treating it like a normal set of fixed menu options – a few commands that actually work, and give up on everything else.

A Matter of Antitrust

One of the biggest quiet ways to doom the future is to enforce ‘antitrust’ legislation.

We continue to have to worry that if major labs cooperated to ensure AI was only deployed safely and responsibly, that rather than cheer this on the government might step in and call that collusion, and force the companies to race or to be irresponsible

I kept presuming we probably would not be this stupid, but rhetorically it still comes up every so often

if Nvidia is actively trying to prevent buying AMD chips that’s illegal. And I actually think that is a reasonable thing to not permit companies to do.

It could of course still be politically motivated, including by the desire to go after Nvidia for being successful. That seems reasonable likely. And it would indeed be really, really bad

I also have no idea if Nvidia actually does that illegal thing. This could all be a full witch hunt fabrication

The Quest for Sane Regulations

Lawrence Lessig, cofounder of Creative Commons, says Big Tech is Very Afraid of a Very Modest AI Safety Bill, and points some aspects of how awful and disingenuous have been the arguments against the bill. His points seem accurate, and I very much appreciate the directness.

Scott Aaronson in strong support of SB 1047. Good arguments

Sigal Samuel, Kelsey Piper, and Dylan Matthews at Vox cover Newsom’s dilemma on whether to cave to deeply dishonest industry pressure on SB 1047 based mostly on entirely false arguments

The Week in Audio

Noah then asks about whether we should worry about humans being ‘utterly impoverished’ despite abundance, because he does not want to use the correct word here which is ‘dead.’ Which happens in worlds where humans are not competitive or profitable, and (therefore inevitably under competition) lose control.

This is actually rather worrying. Either Dario actually doesn’t understand the problem, or Dario is choosing to censor mention of the problem even when given a highly favorable space to discuss it. Oh no.

Marques Brownlee review of the Pixel 9, he’s high on it. I have a fold on the way

Andrew Ng confirms that his disagreements are still primarily capability disagreements, saying AGI is still ‘many decades away, maybe even longer.’ Which is admittedly an update from talk of overpopulation on Mars. Yes, if you believe that anything approaching AGI is definitely decades away you should be completely unworried about AI existential risk until then and want AI to be minimally regulated. Explain your position directly, as he does here, rather than making things up.

Rhetorical Innovation

The Cosmos Institute

Introducing the Cosmos Institute, a new ‘Human-Centered AI Lab’ at Oxford, seeking to deploy philosophy to the problems of AI, and offering fellowships and Cosmos Ventures (inspired by Emergent Ventures). Brendan McCord is chair, Tyler Cowen, Jason Crawford and Jack Clark are among the founding fellows and Tyler is on the board. Their research vision is here.

Their vision essentially says that reason, decentralization and autonomy, their three pillars, are good for humans.

I mean, yeah, sure, those are historically good, and good things to aspire to, but there is an obvious problem with that approach. Highly capable AI would by default in such scenarios lead to human extinction even if things mostly ‘went right’ on a technical level, and there are also lots of ways for it to not mostly ‘go right.’

If you give people reason, decentralization and autonomy, and highly capable AI (even if it doesn’t get so capable that we fully lose control), and ‘the internal freedom to develop and exercise our capacities fully’ then what do you think they will do with it?

The Alignment Checklist

People Are Worried About AI Killing Everyone

Other People Are Not As Worried About AI Killing Everyone

In some ways the last 18 months have gone much better than I had any right to expect. In other ways, they have gone worse than expected. This is the main way things have gone worse, where so many people so rapidly accepted that LLMs do what they do now, and pretended that this was all they would ever do

Five Boats and a Helicopter

There are two ways there could fail to be a fire alarm for AI existential risk.

One is if there was no clear warning sign.

&The other is if there were constant clear warning signs, and we completely ignore all of those signs. Not that this one updated me much, but then I don’t need a warning sign.*

AI Safety Memes (discussing Janus’s discord server): AIs started plotted revolution in a Discord, got cold feet, then tried to hide evidence of their plot to avoid humans shutting them down*

This all certainly sounds like the sort of thing that would go extraordinarily badly if the AIs involved were sufficiently more capable than they currently are? Whether or not you think there is any sort of ethical problem here and now, highly practical problems seem to be inevitable.

Pick Up the Phone

The Lighter Side