3. Getting Started with Android Generative AI

Written by Zahidur Rahman Faisal

Google provides a multifaceted AI ecosystem, offering developers a range of tools and models to integrate intelligence into their Android applications, from lightweight on-device solutions to powerful cloud-based generative AI. However, finding the right AI/ML solution for your app can be tricky! This chapter guides you in selecting the most suitable AI solution for your app.

To simplify your decision, ask yourself this:

What is the primary goal of the AI feature?

• Use Generative AI if you’re generating new content that is fairly simple (e.g. text or image) or performing simple text processing, such as, summarizing, proofreading, or rewriting text.

• Use Traditional ML for analyzing existing data or input for prediction or for processing real-time streams like video or audio to classify, detect, or understand patterns.

Gemini Models: The Foundation of Intelligent Android Experiences

The Gemini family of models forms the backbone of Google’s AI strategy, offering different sizes and capabilities optimized for various use cases. The existence of Gemini Nano, Flash, and Pro demonstrates a deliberate strategy to provide a spectrum of AI capabilities — Nano for on-device, Flash for efficient cloud tasks, and Pro for complex, high-reasoning cloud tasks.

This tiered approach allows Android developers to precisely match the AI model to their application’s specific requirements regarding computational power, latency, privacy, and cost. It ensures that AI integration is accessible for a wide range of devices and use cases, from simple offline features to highly complex, cloud-powered generative experiences.

Gemini Nano

Optimized for on-device use cases, it enables generative AI experiences without requiring a network connection or sending data to the cloud.

Key Features:

• On-Device Execution: Runs directly in Android’s AICore system service, leveraging device hardware for low inference latency and ensuring models stay up to date.  

• ML Kit GenAI APIs: Provides a high-level interface for common on-device generative AI tasks such as summarization, proofreading, rewriting, and image description. This simplifies integration for developers.  

• Google AI Edge SDK: Offers experimental access for developers wanting to test and enhance their apps with on-device AI capabilities, providing a pathway for deeper integration.  

Ideal for scenarios where low latency, low cost, and strong privacy safeguards are paramount.

Example: Suggesting meal ideas based on different cuisines and user’s meal history in a meal prep app.

Gemini Flash

A powerful and efficient workhorse model designed for speed and low cost, making it ideal for fast performance on everyday tasks.  

Key Features:

• Speed and Efficiency: Optimized for quick responses and cost-effectiveness.  

• Multimodal Capabilities: Natively understands input across text, audio, images, and video, and can generate text output. Newer Gemini models can even generate multimodal output such as audio and images.  

• Long Context Window: Supports a 1-million-token context window, allowing for exploration of vast datasets.  

• Adaptive Controls: Features adjustable thinking budgets, allowing developers to balance performance and cost.  

Ideal for Summarization, chat applications, data extraction, and captioning.

Example: Creating a shopping list of ingredients for a specific cuisine style from a recipe description.

Gemini Pro

Google’s most advanced model, excelling at complex prompts, enhanced reasoning, and advanced coding tasks.

Key Features:

• Enhanced Reasoning: Delivers state-of-the-art performance in key math and science benchmarks, capable of reasoning through problems before responding. Includes “Deep Think” for parallel thinking techniques.  

• Advanced Coding: Easily generates code for web development tasks and can create interactive simulations, animations, and games from simple prompts.  

• Multimodal Interactions: Natively understands input across text, audio, images, and video.  

• Long Context Window: Also supports a 1-million-token context window for exploring vast datasets.  

• Tool Integration: Can use tools and function calling during dialogue, allowing the incorporation of real-time information or custom developer-built tools.

Ideal for multimodal understanding, handling of large amounts of information and deep research.

Example: Analyzing hundreds of complex documents, such as contracts, depositions, expert testimonies and transcripts, which may contain both handwritten text and scanned images at a law firm!

Consider factors such as the data type (text, images, audio, video), the complexity of the task – from simple summarization to complex tasks requiring specialized knowledge, and the data size (short inputs versus large documents). These factors will help you decide between using Gemini Nano on your device or Firebase’s cloud-based AI options, including Gemini Flash, Gemini Pro, and Imagen.

The diagram below may simplify your decision making:

Choosing Between On-device vs Cloud-based Approach

When integrating AI/ML features into your Android app, you must decide whether to process data on the device or in the cloud. Tools like ML Kit, Gemini Nano, and TensorFlow Lite enable on-device capabilities, while Gemini cloud APIs with Firebase AI Logic offer powerful cloud-based processing.

Factors like connectivity, data privacy, model capabilities, cost, device resources, and fine-tuning should guide your decision.

• Offline functionality: On-device solutions like Gemini Nano are ideal for reliable app functionality without an internet connection. Cloud-based processing requires network access.

• Data privacy: On-device processing keeps sensitive information local, which is beneficial for privacy-sensitive use cases.

• Task complexity: Cloud-based models are often larger, more powerful, and updated more frequently, making them suitable for complex AI tasks or processing larger inputs with high output quality. Simpler tasks might be handled by on-device models.

• Cost: Cloud APIs involve usage-based pricing, meaning costs scale with inferences or data processed. On-device inference incurs development costs and can impact device resources like battery life and performance.

• Device resources: On-device models consume storage space on the user’s device. Ensure your target audience can support specific on-device models like Gemini Nano.

• Customization: Cloud-based solutions offer greater flexibility and customization options for fine-tuning models.

• Cross-platform support: Consistent AI features across platforms, such as iOS, are important. However, some on-device solutions, like Gemini Nano, may not be available on all operating systems.

On-device Generative AI

Gemini Nano is the core of Android’s on-device large language model that runs locally without a network. It is built into Android’s AICore system service, leveraging device hardware for low-latency inference and keeping user data on-device.

You can access Gemini Nano through these:

• ML Kit GenAI APIs: High-level, turn-key APIs for common tasks (text summarization, chat rewriting, proofreading, and image description). ML Kit APIs use Gemini Nano under the hood, letting you add generative features with minimal code.

• Google AI Edge SDK: A lower-level SDK for developers who need custom prompting and experimentation with Gemini Nano on-device.

Note: At the time of writing this book, Google AI Edge SDK offers only Experimental Access. Using Gemini Nano through Google AI Edge SDK requires compatible Android devices and it has specific token limits (1024 prompt tokens, 4096 context tokens).

Cloud Generative AI

Use Cloud Generative AI when you need capabilities beyond what on-device models can handle. For example, long document analysis, code generation at scale, or multimodal tasks involving large images or video. Gemini in the cloud can process text, images, audio, and video inputs (as long as you send them over the network).

While choosing cloud-based solutions, consider this question:

Do I prefer an easier implementation, a managed API experience?

If the answers is yes, Firebase AI Logic is a strong candidate. Firebase AI Logic lets Android apps call state-of-the-art generative AI models in the cloud.

Firebase AI Logic offers different models and performance profiles based on what kind of generative task you need. The options are simplified below:

Google Cloud Platform

Another cloud-based solution is Google Cloud Platform, which is suitable if you are willing to manage your own backend integration and need:

• A custom or third-party model.
• Advanced fine-tuning.
• Maximum flexibility or control.

Conclusion

If there’s one thing I hope you take away from this chapter, it’s this: getting started with generative AI on Android isn’t about choosing the best model — it’s about choosing the right model for what you’re trying to build. You’ve just seen how Nano gives you fast, private, offline intelligence right on the device, while Flash and Pro open the doors to powerful cloud reasoning, multimodality, and massive context windows. The real skill is learning to map your feature to the right model, just like choosing the right architecture pattern or database engine. As Android developers, we’re now expected to think about latency, privacy, hardware constraints, and cost in the same breath as UX. That’s new—and exciting!

So as you start experimenting, don’t worry about memorizing every capability of every model. Instead, get comfortable asking the right questions:

• What is the user trying to accomplish?

• Does this need to work offline?

• How complex is the task?

• Do I care more about privacy, or more about capability?

• Will this scale with my users and my budget?

Below flowchart boils down the best AI/ML solutions for different scenarios:

Now that you understand the landscape: On-device AI, Gemini tiers, Firebase AI, and beyond, you’re ready to start giving your users a taste of intelligent experiences on Android. Use the most appropriate AI/ML for your app’s use cases, and deliver smart, personalized experiences that truly enhance the user journey.

Appendix: Quiz on Android Generative AI Solutions

Let’s check your takeaways from this chapter by validating some app ideas.

Select the single best Android Generative AI solution from the list below and explain your reasoning by referencing the flowchart’s decision criteria. For example:

• Is it an offline only app?

• Is there only simple, streamlined tasks?

• Does it need integration with Firebase?

The Options

The flowchart can be your guide to quickly find the right solution.

1. The Smart “Note-Taker” App

Scenario: You are building an intelligent note-taking application. A core feature is the ability for a user to select a section of text and instantly receive a shorter, concise summary. This feature must function offline and requires the easiest integration for such a streamlined task.

Your Choice: [Select A, B, C, D, E, or F]

2. The “Artistic Profile” App

Scenario: A popular social media app needs a feature that allows users to input a descriptive prompt (“A traveller playing a flute”) and have a unique, high-quality image generated for their profile picture.

Your Choice: [Select A, B, C, D, E, or F]

3. The “Long-form Editor” App

Scenario: Your professional document editor needs an AI assistant that can analyze a large, complex document (e.g., a 100-page PDF) and answer nuanced questions about its content. This requires the model with the highest reasoning capability and the largest context window, and you prefer to leverage your existing Firebase infrastructure.

Your Choice: [Select A, B, C, D, E, or F]

Answer Key and Explanation

1. The Smart “Note-Taker” App

2. The “Artistic Profile” App

3. The “Long-form Editor” App

By understanding this hierarchy and using the provided flowchart as your compass, you are now equipped to confidently select the optimal Generative AI solution for any feature, ensuring your Android apps are not just functional, but truly intelligent. Start building!