At Google I/O 2024, Google has shown off a lot of new improvements coming to Android, thanks to Gemini. With Gemini, Android will be aware of the context on the screen in a variety of scenarios, which will make your life a lot easier. At least, according to Google.
Circle to Search, which was first shown off during Samsung’s Galaxy Unpacked event earlier this year, will be getting some new enhancements. Now, Circle to Search will be a great new study buddy for students. Why? Because it will be able to help with homework, like physics and math. You can simply circle a prompt on the screen, and it will give you step-by-step instructions on how to solve it.
This new feature for Circle to Search will be available today. By the end of the year, it should be available on over 200 million devices.
A major focus of Gemini is how it will be context-aware of what’s on your screen. For example, Gemini can generate an image related to a text message, and it can be dropped right into the conversation. You can also ask Gemini specific questions about a video that you’re currently watching or even long PDF files so you don’t have to scroll through multiple pages. This feature will be rolling out to devices over the next few months.
Gemini Nano multimodal capabilities will be available for Google Pixel later this year. With this feature, your smartphone will be able to process text, as well as understand more information in certain contexts such as sounds, sighs, and spoken language.
An accessibility feature that’s being improved with Gemini is TalkBack. This is a feature that is designed for people who experience blindness or low vision. TalkBack allows users to get richer and clearer descriptions of what is happening in an image. This Gemini update to TalkBack will fill in more missing information, like details about what is in a family photo or even the cut of clothes when shopping online. No data connection is needed since Gemini Nano is on-device.
Gemini will also further help prevent scam calls. Gemini Nano can provide real-time alerts during a call if it is able to detect conversation patterns that are commonly associated with scams. For example, when a “bank representative” asks you to transfer funds, make a payment with gift cards, or provide personal information.
When Gemini Nano detects these phrases, it will give you an alert on the screen while you’re on the call with a bit of information, like telling you that banks will never ask you to move money.
This is just the start of improvements coming to Android 15.
