What’s around the corner: Predictions from the research that sparked the current AI explosion

Two co-authors of the seminal “Attention is All You Need” paper as well as cofounders of Essential AI, Niki Parmar and Ashish Vaswani, share their perspectives with Managing Director Ganesh Bell on the next big challenges — and solutions — for artificial intelligence progress. What are the biggest areas of potential, and who is best positioned to win the future?

These insights came from our ScaleUp: AI event in October 2023, an industry-leading global conference that features topics across technologies and industries. Watch the full session below:

Transformer model’s founding moment and its future

The co-creators of the Transformer model discussed the founding moment of their revolutionary AI. They emphasized that while they were aware they were working on a fundamental problem, they didn’t fully realize the potential of their innovation until they saw the model outperforming existing ones and applying it to various tasks.

“When these things happened in quick succession, and you’re actually seeing not just one task but multiple tasks in multiple languages, we knew that this was a very, very powerful general system that we had.”

Vaswani explained, “We started working on the Transformer because we wanted to both read text all at once, which is what the transformer does. But we also wanted to produce all the words at once. When these things happened in quick succession, and you’re actually seeing not just one task but multiple tasks in multiple languages, we knew that this was a very, very powerful general system that we had.”

Parmar also acknowledged the surprising rapid adoption and continuous development of the Transformer model: “The quick adoption of it and the ways in which people are pushing it forward was definitely extremely surprising, and it’s also inspiring at the same time because now we can all collectively do a lot more.”

The role of open source in the advancement of AI

The speakers shared their thoughts on the potential role of open source in AI’s future. They discussed the need for a community-built manual on training an open-source model and the importance of data accessibility in this field.

“Where can people contribute the most? Data keeps coming up because it’s such unknown today.”

Vaswani highlighted the need for contributions from people, “Where can people contribute the most? Data keeps coming up because it’s such unknown today. There [are] also open source interpreters that actually use closed source models and open source models. So I think that there are several places where we can have large open source contributions.”

Parmar, on the other hand, emphasized on the need for better evaluations in open source AI, “One thing that keeps coming up is the need for better evaluations. We’re relying on existing very handcrafted and not very general evaluations. Open source does have a chance in bringing about standard methodology around how to evaluate.”

AI’s potential to solve previously impossible problems

The session closed with a discussion on AI’s potential in solving previously thought impossible problems. The speakers expressed their hopes for AI’s role in education and solving significant global issues.

Parmar expressed her excitement about the possibilities of AI in education, “For me personally I think I’d be really, really excited if these systems can really provide easy learning and easy access to education for everyone.”

“For me personally I think I’d be really, really excited if these systems can really provide easy learning and easy access to education for everyone.”

Vaswani shared a similar sentiment, looking forward to the solving of significant global issues with the help of AI, “I would like to see some of the Millennium Problems solved in conjunction with the new AI models.”