Meta says smaller is better

Author

Calin Drimbau
February 27, 2023

polar bear

It’s International Polar Bear today! Bears absolutely no significance to today’s content, but I thought you should know. I love Polar Bears.

What do we have for you today?

🎻 Meta joins the AI race: Meta launched a set of LLMs with claims that their star model can outperform GPT-3, despite being 10x smaller

🐰 How deep does the rabbit hole go on ethical sourcing of training data? Stability AI’s CEO hints at private data being used in the training of models

🎚️ Yo, DJ, spin that wheel! Spotify launches new “DJ in a pocket” feature. Is this the start of personalized radio?

Meta says smaller is better

smaller

Meta announced the release of LLaMA (Large Language Model Meta AI). The release is aimed at researchers and Meta is making it available on a case-by-case basis.

The catch? It’s not just one model. It's actually a herd of LLaMAs (see what I did there?), and according to Meta AI researchers, some of their models can outperform OpenAI's GPT-3 despite being smaller. In particular, the star LLaMA, LLaMA-13B, reportedly outperforms ChatGPT-like technology despite being 10 times smaller.

Meta has made the entire code and weights of their models publicly available. In contrast, the models underlying OpenAI's ChatGPT and Google's LaMDA are not publicly available.

The release made AI researcher Simon Willison say: "I'm now thinking that we will be running language models with a sizable portion of the capabilities of ChatGPT on our own (top of the range) mobile phones and laptops within a year or two”. Qualcomm also demo’ed last week what they claim to be the fastest-ever deployment of AI image generator Stable Diffusion on a smartphone.

I’m thinking: better start training those LLaMAs soon.

Hacking for data

private data

The evolution of large language model capabilities is predicated on two things: the models themselves are improved and more data is fed as training into the models.

Emad Mostaque, Stability AI’s CEO claims we’re about to see another jump in the capabilities of LLMs for image generation, but the Tweet sparked a debate on when we will see all of these developments plateauing.

In response to a suggestion that we’ll see smaller leaps once there is no new data to train the models on, Emad suggested one could go behind firewalls to get private data.

If we start going “Mr. Robot” on this, I’m fairly sure AI data training audits aren’t a thing yet. We’re yet to see the EYs and KMPGs descending into AI company offices looking for that private illegal data that one has “borrowed” from behind firewalls.

And even if they do, AI is a black box, training data can be deleted, and no one will know what data went into the black box. How’s that for a new ethical dilemma?

Finally, a DJ that listens

dj

It was only a matter of time before Spotify joined the race to integrate new AI capabilities into their products. Last week, Spotify announced a feature called “DJ” that will better personalize the listening experience.

The feature will provide a curated selection of music, accompanied by AI-powered spoken commentary about the tracks and artists you enjoy. Spotify claims that the voice used for the spoken commentary is "stunningly realistic," similar to that of a radio DJ.

Spotify’s been doing relatively well on the personalization front, so it will be interesting to see the wider reactions to the new feature release. I wonder if this is the start of personalized AI radio. A sensible set of music recommendations with commentary in between that is valuable to me as a listener? Sounds like a radio killer. If anyone is still listening to radio, that is.

Gen AI Deals that make your eyes (and mouth) water 💰

What caught our eye? 👀Voicemod is joining the new wave of startups focused on enabling users to create completely new synthesized voices and even use these voices in real-time. The next generation of karaoke will enable us to transform our own voice into our favorite artist’s voice!

What caught our eye? 👀Spoke.ai aims to reduce workplace noise for knowledge-driven workers by using a combination of natural language processing and generative AI models.

This AI summarization tool integrates with popular interfaces like Slack, JIRA, and others to produce a live record of must-see workflows that includes support tickets and documents. It is intended to serve as an always up-to-date source of truth in summarized form.

New developments to spam your #random Slack channel 💬

Things to learn when you need a raise

Want access to the Generative AI investors database? Share and subscribe to AI In the Middle (below) to receive it! ⬇️

Before you go

Let the logos come to life with ControlNet.

And scene. That’s all for this week, we’ll see you Thursday. Thanks for reading, sharing, and subscribing. Have something to share? Slide in our DMs.

— Calin Drimbau (@calindrimbau)