AI is everywhere in children's apps now. Story generators that write personalised bedtime tales. Drawing tools that animate a child's sketches. Tutoring assistants that explain maths in a friendly voice. These tools can be wonderful for learning and creativity. But there's a critical question most parents never think to ask: where is the AI actually running?

The Rise of AI in Children's Apps

The explosion of generative AI since 2023 has reached the kids' app market at full speed. AI features have spread quickly through children's apps, from story generators and drawing tools to homework helpers. They range from simple text completion to sophisticated image generation and voice interaction.

For children, AI can be genuinely transformative. A shy child who won't read aloud to a parent might happily narrate stories to an AI companion. A child struggling with fractions can get patient, adaptive explanations that adjust to their specific misconceptions. The potential is real.

But the architecture behind these features — specifically, whether the AI runs in the cloud or on the device — has profound implications for your child's privacy and safety.

How Cloud AI Works

When an app uses cloud-based AI, here's what happens every time your child interacts with it:

• Your child types a prompt, speaks a question, or draws a picture
• That input is packaged into a network request and sent to a server — typically operated by the app developer or a third-party AI provider like OpenAI, Google, or Anthropic
• The server processes the input using a large AI model
• The result is sent back to the device and displayed

This means the child's actual content — their words, their questions, their drawings, their voice — physically leaves the device and travels to a computer in a data centre somewhere. In many cases, this data is logged, stored, and potentially used to train future AI models.

What Exactly Goes to the Server

This is worth being specific about, because "data" sounds abstract. Here's what cloud AI systems typically receive from a children's app:

• The child's exact text input. If your child types "I'm scared of the dark and my name is Emma and I live on Maple Street," that entire string goes to the server.
• Voice recordings. If the app uses speech-to-text, the raw audio is often transmitted for transcription. This is a recording of your child's voice.
• Drawings and images. AI drawing assistants that animate or modify sketches typically send the image file to a server for processing.
• Conversation history. To maintain context, many AI systems send the full conversation history with each new request. A 20-minute tutoring session generates a detailed transcript of everything the child said and every mistake they made.
• Metadata. Device identifiers, approximate location, timestamps, app usage patterns.

Even if the app developer has good intentions, this data now exists on their servers, subject to their security practices, their data retention policies, their country's legal framework, and the policies of whatever third-party AI provider they've integrated.

Why This Matters More for Children

Adults make informed trade-offs about data sharing every day. Children cannot, for several reasons:

• COPPA and legal protections. The Children's Online Privacy Protection Act (US) and the UK Age Appropriate Design Code exist precisely because children's data requires special handling. Verifiable parental consent is required before collecting personal information from children under 13. Yet enforcement is inconsistent, and many apps skirt these requirements.
• Data permanence. A child's data collected today will exist long after they grow up. An embarrassing question a 7-year-old asks an AI tutor could theoretically persist in a training dataset for decades. Children cannot consent to this long-term implication.
• Content moderation challenges. Cloud AI systems must prevent children from encountering harmful content. This is extremely hard to do perfectly. Prompt injection attacks, hallucinated inappropriate content, and edge cases in content filters are ongoing challenges even for the best AI labs.
• Breach vulnerability. When children's data is aggregated on a server, it becomes a target. The 2015 VTech breach exposed personal data of 6.4 million children. Data that doesn't exist on a server can't be stolen from one.

How On-Device AI Is Different

On-device AI — sometimes called "edge AI" — runs the AI model directly on the phone or tablet. The child's input never leaves the device. Here's the flow:

• Your child types, speaks, or draws
• The input is processed by a model stored locally on the device
• The result is generated on the device and displayed
• Nothing is transmitted. No server is involved.

This architecture doesn't just add a privacy feature — it eliminates entire categories of risk. There's no server to breach. No conversation logs to subpoena. No training data pipeline to accidentally include children's content. No third-party AI provider with their own data policies.

The Trade-Offs Are Real

On-device AI isn't simply "cloud AI but private." There are genuine trade-offs:

• Model size. The largest cloud models are many times bigger than anything that fits on a phone; on-device models typically have a few billion parameters. They're less capable at complex reasoning, nuanced language, and multi-step tasks.
• Response quality. A cloud-based story generator will produce more creative, more coherent, and more varied stories than a comparable on-device model. The gap is narrowing, but it's still real.
• Hardware requirements. Running AI locally requires modern hardware. Older phones and tablets may lack the processing power or memory for on-device inference. Recent Apple and Qualcomm chips handle this well, but budget devices may struggle.
• No real-time knowledge. Cloud models can (in principle) access current information. On-device models only know what they were trained on.

For children's apps, however, these trade-offs are often acceptable. A story generator for 6-year-olds doesn't need the largest cloud model. A maths tutor for primary school doesn't need real-time internet access. The bar for "good enough" in children's content is different from adult applications.

The On-Device Trend

Phone and tablet makers have invested heavily in making on-device AI practical. Recent iPhones and iPads include chips designed specifically for machine learning, with performance that would have needed a server only a few years ago.

Apple Intelligence, introduced in 2024, reinforced this direction. Apple's explicit positioning is that personal data should be processed on-device wherever possible, with cloud processing used only when necessary and protected by additional cryptographic guarantees (Private Cloud Compute).

Google has made similar moves with on-device processing in Android, and the broader trend is clear: the industry is moving toward keeping sensitive data on the device.

What Parents Should Look For

When evaluating AI-powered apps for your children, ask these questions:

• Does it work offline? Try the airplane mode test. This is the single most reliable indicator of on-device processing.
• What does the privacy policy say about AI? Look for explicit statements about whether AI inputs are transmitted, stored, or used for training. Vague language like "we may process data to improve our services" is a red flag.
• Who provides the AI? If the app uses a third-party AI API (OpenAI, Google, etc.), your child's data is subject to that third party's policies as well as the app developer's.
• Is there a kids' privacy certification? Look for kidSAFE, PRIVO, or similar certifications that specifically audit children's data practices.
• What data does the App Store privacy label show? Apple requires developers to disclose data collection. Check the "App Privacy" section for any app before downloading.

Doodlebee is one example: a family creative app whose optional AI story helper runs entirely on-device, on supported devices. Its colouring, drawing, animation, storybook and music tools all work fully offline, with no data leaving the device. But regardless of which apps you choose, the airplane mode test works for any of them.

The Bigger Picture

This isn't about being anti-AI. AI-powered tools can genuinely help children learn, create, and explore. The question is whether that help requires sending a child's creative output, questions, mistakes, and personal details to a server.

For adult applications where users can make informed consent decisions, cloud AI is often the right trade-off. For children, who cannot meaningfully consent and whose data deserves the highest protection, on-device AI offers something cloud AI architecturally cannot: the guarantee that private data stays private, not through policy promises, but through the simple fact that it never leaves the device in the first place.