The biggest Google I/O announcements from Gemini to AI and search

August 10, 2024

MOUNTAIN VIEW, Calif. — In speeches and demonstrations at the company’s annual developer conference on Tuesday, Google executives showed off a vision for its future, where artificial intelligence helps people work, plan their lives, navigate the physical world and get answers to questions directly. It would change the way the internet works forever.

In the biggest overhaul to Google’s search engine in years, the company said it will roll out AI-generated answers to the top of everyone’s search results in the United States this week, and to a billion of its worldwide users by the end of the year.

It also pushed its new and improved voice assistant that can answer questions more skillfully than before. Instead of connecting people to the broader web, Google’s AI will now do the reading and researching for them, summarizing websites, videos and social media posts into “overviews” that include everything they need to know on any given topic.

“Google will do the searching, the researching, the planning, the brainstorming and so much more. All you need to do is just ask,” Elizabeth Reid, Google’s head of search, said onstage.

In one example, an executive asked Google’s Gemini assistant to plan a trip to Miami for her and her family. The AI searched the internet, reading reviews and travel guides written by humans, and put together an itinerary. The company showed off dozens more examples, from helping people learn how to flirt, to giving a suggestion for a last-minute gift.

The tsunami of new AI features come as the tech giant has thrown tens of billions of dollars into building AI tools to respond to competition from Meta, Microsoft, ChatGPT-maker OpenAI and a host of up-and-coming AI start-ups. AI features will prominently be displayed across Google’s products, including Google Docs, Google Photos, Gmail and YouTube.

Google researchers invented many of the core technologies that kicked off the AI arms race, but over the past year the company has been on its back foot, with many in the industry seeing its tech as lagging behind that of OpenAI. On Tuesday, the company sought to prove it is still the king of the AI world, showing off improvements to its core AI model, which it calls Gemini.

Outside the conference, which takes place at an open-air amphitheater near Google’s headquarters, pro-Palestinian protesters gathered to demand the company end its work with Israel’s government and military. In April, Google fired 50 workers for holding sit-ins at the company’s offices to protest its contract with Israel.

Here are the biggest announcements from the company.

AI answers take over search

Google is making the biggest changes to its search engine since it launched its core product over 20 years ago. Now, instead of showing links to other sites or snippets of those sites at the top of search results, the company will use AI to summarize websites and provide multi-paragraph answers to search queries.

The changes have been in public testing for a year, but this week Google confirmed that it would aggressively push it to its hundreds of millions of users in the United States and further abroad, whether they want to use it or not. The changes are part of a broader vision outlined by Google CEO Sundar Pichai, in which Google will be the central hub of how information is accessed for everyone. The company will ingest social media comments, online videos and news articles and remix the information using AI, spitting it out again in whatever format its users want.

Publishers are warning the changes could devastate their businesses, as more people find their answers directly on Google and don’t click through to the source of the information. Google says it doesn’t want to damage the open web and that it is still prioritizing sending traffic to websites. Users can’t turn off the AI answers, even if they want to.

AI is still far from ready to answer every question well. Even Google’s slick, highly-produced promotional video had an error where it instructed someone to fix a camera in a way that would expose and damage the film.

Google’s AI bot Gemini gets smarter

Google’s flagship AI model — its answer to OpenAI’s GPT4 — is called Gemini. The company demonstrated its capabilities, like showing it a bookshelf through a phone camera and getting it to quickly make a spreadsheet of all the books and their authors. In briefings before the event, Google showed a video of an employee walking through an office with a phone camera open, asking Gemini questions. The AI analyzed computer code on a workstation monitor, looked out the window and identified the neighborhood the person was in and even made up a clever name for a band consisting of the office golden retriever and a stuffed tiger toy — “Golden Stripes.”

The improved version of Gemini is available to all developers around the world, and to consumers who pay for an advanced version of Google’s AI app.

The day before, OpenAI had showed off a similar tool, asking its own AI chatbot to describe a room and the activities of the people in it.

Google also said that Gemini could now take in more complex instructions. For example, a student could upload an entire thesis paper and ask for feedback or ideas on how to change it.

Google’s head of AI, Demis Hassabis, also teased the company’s Project Astra. It is Google’s effort to build an AI “agent” that could do tasks for people by navigating the web on its own. Theoretically, AI agents could do things like book dentist appointments, communicate with colleagues on your behalf, and research places to eat and make a reservation.

A new AI video tool, Veo

Generative AI companies, including Google, want to revolutionize the way people create visual images, audio and movies. At I/O, Google announced a new video-generating AI tool called Veo, which aims to compete with OpenAI’s Sora. Veo generates high definition videos that can be longer than a minute, a threshold Google had yet to achieve.

Before the big speeches, DJ Marc Rebillet tried to warm up the crowd by making beats using Google’s AI tools. Rebillet bounced around the stage yelling “Google” over and over again. Google said it is working with creators including Rebillet, musician Wyclef Jean, and actor and producer Donald Glover on AI creations.

Google also showed off a new image-generation AI tool called Imagen 3, meant to compete with OpenAI’s Dall-E 3. The tech allows people to generate realistic-looking images with text prompts.

Work apps get even more AI

Google has been putting AI features into its suite of productivity apps including Gmail, Docs, Drives and Sheets over the past year. At I/O, the company announced some new tweaks, allowing users to summarize groups of emails from the same sender, adding details from a Google Doc in an email or incorporating content from a spreadsheet into a Slides presentation.

The company will also begin letting people ask Google’s AI to find specific details in a document and add them to an email. Google’s “help me write” feature, which generates text from scratch, will also soon be available in Spanish and Portuguese.

Google showed how its Gemini AI tool can also be used to teach kids about new concepts, asking it to explain the physics behind how a basketball rolls and bounces.

Android wants to catch scam calls

Google owns the Android smartphone operating system, which runs on the majority of phones worldwide. The company is trying to make Android more appealing than Apple’s iOS by putting more AI into the operating system itself. One improved feature, called Circle to Search, allows a person to circle anything they have a question about or want more information on and immediately get search results. The user can also generate images for text messages by asking Gemini.

Gemini can also help users get information from videos and PDFs. While they’re watching a video, for example, they can ask a specific question about something that happened in it. When they ask a question about a PDF, it’ll refer users to the part of the PDF where it found the answer.

Scam calls have become an even bigger problem as AI voice generators allow fraudsters to mimic real people. Android previewed a feature that will listen to and interrupt calls with a notification to the user if it thinks the call is coming from a scammer, such as if the caller asks for bank account information.

correction

In a previous version of this article, the caption for the top photograph incorrectly said it was of the 2023 I/O conference. The photograph was taken Tuesday. The caption has been corrected.

ncG1vNJzZmivp6x7uK3SoaCnn6Sku7G70q1lnKedZMGmr8enpqWnl658c3yRbWZpbV9mgXCzzqiepZ1dnrxurc2npq6mk5q6prrTrGRraGJpfA%3D%3D