Galaxy AI: Here's what Samsung-Google's gen AI feature is capable of doing on Galaxy S24 series

Galaxy AI promises deliver the best on-device user experience and improve productivity compared to rival premium branded phones in the market.

Samsung on Wednesday (January 17) unveiled the Galaxy S24 series, the company's new premium smartphone variants.

It comes in three variants-- Galaxy S24, S24 Plus and S24 Ultra. They come with big upgrades in terms of camera, processor and more.

The USP of the new Galaxy S24 series is the generative Artificial Intelligence (gen AI)-powered Galaxy AI feature. It promises to deliver the best user experience compared to rival premium branded phones in the market.

Galaxy AI: Here are key aspects you should know about the new-gen AI feature of the S24 series

At the launch event, there was little confusion on what is powering the Galaxy AI. Is it Google Cloud's Large Language Model (LLM) Gemini Nano or Gemini Pro? Now, we have all the details. Both Google's LLM models will drive the Galaxy AI based on the requirements of a particular task.

For tasks that can be done on-device without the need for an internet connection, Gemini Nano-based Galaxy AI will do the job. And, Gemini Pro will come into play for the heavy-duty tasks that need cloud-based server connectivity.

Google Gemini 1.0 Large Language Model announced.

Here are Galaxy AI's on-device generative AI capabilities

Live Translate: It can perform two-way, real-time voice and text translations of phone calls within the native phone app on the Galaxy S24 series phones. This way, there will be no need for third-party apps and on-device AI keeps conversations completely private.

Interpreter: This feature can instantly translate live conversations on a split-screen view so people standing opposite each other can read a text transcription of what the other person has said.

Google DeepMind's Gemini comes in three forms-- Ultra, Pro and Nano.

Here are Galaxy AI's capabilities that need an internet connection

Samsung Keyboard: This too can on-device translation messages in real-time in 13 languages. ( Note: The user has to sign in to a Samsung account).

Android Auto: When a Samsung Galaxy S24 is connected to a compatible car, it can automatically summarize incoming messages and suggest relevant replies and actions, like sending someone your ETA, so the user can stay connected while staying focused on the road.

Circle to Search: It is an intuitive, gesture-driven capability developed by Samsung in collaboration with Google. Users can just circle the subject in a photo or a video in any third-party app. He/she just has to long-press the home button, or users can circle, highlight, scribble on, or tap anything on Galaxy S24’s screen to see helpful, high-quality search results.

For instance, if the user sees a beautiful landmark in the background of a friend’s social media post or a surprising fun fact on YouTube Shorts, he can quickly search to learn more without having to leave that app.

Generative Edit: Here, Gemini Pro will work with the Imagen 2 model on Samsung's Vortex AI, which allows Galaxy S24 series device owners to edit photos. He/she can remove photo bombers, fill in more details in the photo, and also make the image sharper. The latter comes in handy to take Photos taken with digital zoom. They usually contain too much and appear blurry. But, with Generative Edit, it can be made clearer and crisp.

Generative Edit can also fill in parts of an image background with generative AI.

For instance, when a picture is deemed bad, Galaxy AI will fill in the borders. Also, when an object needs to be slightly moved to be in the perfect position, Galaxy AI lets users adjust the position of the subject and generates a perfectly blended background in its original spot.

As a security measure, Galaxy AI will put a watermark anytime Galaxy S24 deploys generative AI to amplify an image. Even the metadata will show that the content was AI-generated.

Note Assist on Samsung Notes app

Note Assist feature in Samsung Notes app: The Galaxy AI can generate summaries, help users build template creation that streamlines notes with pre-made formats, and cover creation to make notes easy to spot with a brief preview.

Custom generate wallpapers: Galaxy S24 series owners can create their own personalised wallpaper and themes with just text prompts. Thanks to advanced Imagen 2's text-to-image engine, the Galaxy S24 can instantly churn out custom wallpapers. For instance, he/she can ask the device to generate a puppy in a beach. It will instantly create a photo with exact subject and scenery in a few seconds.

Galaxy AI editing tools: This will help with simple edits like erasing, re-compose, and remastering even the old photos taken in other devices too.

Edit Suggestion: If the photo lacks any detail or needs remastering, the Edit Suggestion feature uses Galaxy AI to suggest perfectly suitable corrections required for each image.

Instant Slo-mo: The Galaxy S24 series phone will be able to generate additional frames based on movements to smoothly slow down action-packed moments for a more detailed look.

Samsung has revealed that it has plans to bring more powerful Gemini Ultra capabilities to the Galaxy S24 series later in the year.

Read more | Key differences between Google Gemini Nano, Pro, and Ultra

Also, older Galaxy S23, S23 Plus, S23 Ultra, Galaxy Z Fold5, and Z Flip5 will get Galaxy AI later this year.

Get the latest news on new launches, gadget reviews, apps, cybersecurity, and more on personal technology only on DH Tech.