AI-list: Zombies and vampires

AI is a rapidly developing sector that Insight has invested more than $4B since 2014. We’re currently tracking 23,315 AI/ML startups and ScaleUps. Naturally, our team has seen a lot, and the discussions around this topic generate a lot of opinions.

Here’s a look behind the curtain into some of what we’re reading, sharing, and discussing across the team at Insight lately. Think I missed something? Email me and tell me all about it. See the archive of all our AI roundups here.

It’s Halloween in July! Today we’re exploring two big themes in the AI revolution:

1. The challenge for LLMs to continue to be effective on a diet of regurgitated, brain-less “zombie content” generated by AI.
2. The vampiric drain of jobs exacerbated by AI tools in a year of notable tech layoffs.

Queue up The Cranberries and this new banger from Olivia Rodrigo, and let’s get into it.

The dual challenges of “zombie content” and regulation

Last time, I highlighted the ouroboros of AI using AI-generated data to train and tag itself and the resulting challenges. That conversation has since evolved to encompass dual challenges:

1. For LLMs, the challenge that AI models need to feed on high-quality data sources in order to continually generate accurate, usable outputs. At least right now, AI needs to feed on content from human brains in order to remain useful. But what happens when the majority of content LLMs are scraping from the internet is AI-generated? I’m calling this the “zombie content problem.” Experts call it “model collapse.”
2. For humans, the challenge of how and when we regulate this proliferation of AI content. What content should be protected? How can you possibly regulate such a quickly changing space? Lots of new updates here.

Insight from our internal chat: “Sourcing data to feed LLMs will likely be a key battle in the race to build GenAI, with its multiple levels of complexity. More data (as it continues to be generated) is likely key to freshness of the models, but this pushes against increased geographic and regulatory constraints.”

• Via CNN, a new lawsuit against OpenAI for scraping personal data. It claims OpenAI  scraped “massive amounts of personal data from the internet,” and this data was taken by the company without notice, consent, or “just compensation.”
• Via Engaget, Google is responding by updating its policies to explicitly allow the use of “publicly available” data to train its models like Bard, Google Translate, and other cloud projects. These policy updates make it clear that anything you publicly post online could be used to train current and future generative AI products Google develops.
• Via GlobalData, that massive $1.3B Meta fine from the EU for misusing personal data may be driven by concerns about generative AI.
• Meanwhile, via Yahoo Finance, the U.K. is courting a “pro-innovation” stance, openly welcoming tech and venture investment while their EU neighbors draw up legislation regulating it.

What Will AI Do to Your Job? Take a Look at What It’s Already Doing to Coders

Every week when I call my parents, they ask, in slightly hushed voices, if AI has taken my job yet. (For more on what non-tech people think about AI, check out the Verge article below…) The fear isn’t totally unfounded, though: IBM announced a pause on hiring earlier this year and plans to eventually replace many positions with AI. Tech layoffs and a tightening of budgets have been a real trend in 2023, so it’s reasonable to wonder how AI will play a part. Adding additional fuel to the fire: Open AI released Code Interpreter this week.

From the WSJ: “Tech layoffs over the past year, driven by macroeconomic trends, happen to have come just as genuinely useful AI for coding has arrived. For many young coders, the timing is unfortunate. Data from workforce-analytics company Revelio Labs indicate that companies have tended to fire their newest employees, and that in 2023, software engineers represent the largest share of people laid off by tech companies. The few tech job openings that remain, meanwhile, are being snapped up by still-in-demand, more-experienced software engineers.”

But with change always comes opportunity. Might AI lead to entirely new types of jobs?

Google DeepMind’s CEO Says Its Next Algorithm Will Eclipse ChatGPT

The Gemini hype train has left the station. From Wired:

“DeepMind’s Gemini, which is still in development, is a large language model that works with text and is similar in nature to GPT-4, which powers ChatGPT. But Hassabis says his team will combine that technology with techniques used in AlphaGo, aiming to give the system new capabilities such as planning or the ability to solve problems.”

Bits and bots

Turing test yourself. Psychologist and behavioral economist Dan Ariely is running a “Battle of the Wits” challenge between himself and Chat GPT, in addition to this New York Times article testing the same idea for images.

Google’s great “unlearning” challenge to help AI unlearn harmful biases.

A new report highlights the “next generation of AI companies building the future.” (Thanks to Paul for sending this my way!)

What do non-tech people think about AI? How do they use it? Verge surveyed 2,000 people and the results were (surprisingly?) optimistic.

Worth a watch — from April, a 20-minute conversation with Sam Altman on the “breakthrough potential of AI.” Throw it on while you clean out your inbox.

ScaleUp:AI is back October 27

Insight’s premiere AI event, featuring top AI thinkers, doers, investors, and innovators, is back this October. We’re about to announce some of the first speakers in July, and I’m so excited to share more with you and our community. AI-list readers can get *free* virtual tickets by using the code AILIST and registering here.

Editor’s note: Articles are sourced from an ongoing, internal Insight AI/Data Teams chat discussion and curated, written, and editorialized by Insight’s VP of Content and Thought Leadership, Jen Jordan, a mere human vessel capable of digesting and distilling large amounts of information into moderately entertaining formats. 

Image credit: Google Deepmind via Unsplash. Created by Novoto Studio as part of the Visualising AI project launched by Google DeepMind.