Apple Is Letting Your Watch Eavesdrop on You — Carefully

Apple has started beta testing a set of AI features that let the Apple Watch listen to your conversations and hand back notes on what was said, according to TechCrunch. The features, bundled under the name Audio Intelligence, are rolling out to developers as part of iOS 27.3, months after Apple first showed them off at its Surprise and Shine hardware event in September.

There are four opt-in features in the package. Two are fairly uncontroversial: a sound-recognition tool that flags important noises like sirens, alarms, or doorbells for deaf and hard-of-hearing users, and a song-identification feature built on Apple's Shazam technology. The other two are the ones worth paying attention to, because they involve the watch actively listening to the people around you. Apple calls them Live Rewind and Siri Recaps.

Ambient-listening hardware has had a rough couple of years in the court of public opinion. Meta's AI glasses and a wave of similar AI wearables have drawn criticism over the idea of devices passively recording everyone nearby. Apple's pitch with these two features is that it's threading that needle: giving wearers a genuinely useful memory aid without turning the watch into a surveillance device for whoever happens to be standing next to them.

What Live Rewind actually does

Live Rewind is built for the moment you zone out and miss something — a waiter rattling off specials, someone giving you directions, a colleague mentioning a detail in passing. Double-press the digital crown on an Apple Watch Series 12, and the watch shows you a text snippet of whatever was said in the previous 15 seconds, right on the watch face.

From there you have two options. Do nothing, and the snippet disappears for good. Or save it to the standalone Siri app that shipped with iOS 27, where it sticks around for later — handy if you're trying to remember a name or jot down a point from a presentation. There's also an "Ask Siri" prompt that appears below the snippet, letting you ask follow-up questions about something confusing, like an unfamiliar technical term or a menu item you didn't catch.

Apple built in a deliberate signal every time the feature triggers: double-pressing the crown sets off an audible chime and an on-screen animation, meant to make it obvious to anyone nearby that a recording moment just happened. If you don't save a snippet, it's automatically deleted after 30 seconds.

Siri Recaps goes further — but with limits

Siri Recaps takes the same ambient-listening concept and stretches it over a longer stretch of conversation, producing a running summary instead of a 15-second snippet. Apple is explicit about what it won't do here: no speaker identification, no raw audio saved, no word-for-word transcript. Instead, the feature boils a conversation down to a short summary sentence plus bullet points covering things like deadlines, pricing, or other concrete facts.

Apple's suggested use cases lean toward situations where you'd normally be taking your own notes but can't — work meetings, parent-teacher conferences, training sessions, anything where your hands are full or your attention is elsewhere. The framing is that it mimics the notes you'd write yourself, stripped of tone and emotion, not a full record of the exchange. The feature also pauses automatically when you're using headphones for music or noise cancellation.

Nothing saves itself permanently without a nudge from you. At the end of the day, and again when you first unlock the watch the next morning, you're prompted to review your Siri Recaps and decide what to keep. Anything you ignore deletes itself within seven days. What you do keep can be routed to the Notes app or, for more personal entries, the Journal app.

Apple also built in scheduling controls — you can restrict Siri Recaps to work hours, block it from running at night, or just flip it off entirely from the Control Center at any moment, schedule or no schedule. On the content side, Apple says the feature is designed to filter out sensitive categories like government ID numbers, financial and authentication information, other personal identifiers, and language related to self-harm.

The privacy architecture behind it

The engineering underneath these features is where Apple is clearly trying to preempt the inevitable creepiness objections. Audio Intelligence runs on the Apple Watch Series 12 and Apple Watch Ultra 4, using the S11 chip's always-on processor alongside the microphone, Apple Intelligence models on the paired iPhone, and Apple's Private Cloud Compute service.

The S11 chip includes a secure enclave — a hardware-isolated memory space the rest of the operating system can't reach — where a dedicated audio buffer handles raw audio. According to Apple, that audio is never accessible to anyone, including Apple itself, and only reaches the paired iPhone through a secure connection between the two devices.

The actual pipeline works like this: lightweight AI models on the watch's S11 chip detect when the wearer is part of an active conversation, then pass that audio to the iPhone's secure enclave. The iPhone processes it on-device, stripping filler words to produce condensed text. That condensed text gets encrypted and sent to Apple's Private Cloud Compute, which generates the recap and sends it back to the phone — then immediately deletes the recap and any intermediate summaries from its own servers. The finished recaps that land on your iPhone are encrypted in sync across your Apple devices and, per Apple, accessible only to you. Live Rewind's 15-second snippets go through the same watch-to-iPhone handoff for processing.

Apple says none of this is supposed to put a dent in battery life on either the watch or the phone.

For now, these features are limited to developers testing iOS 27.3. Apple plans to widen access to a public beta before Audio Intelligence becomes available to everyone. Whether the privacy engineering is enough to head off the backlash that's dogged other AI wearables will likely depend less on the architecture diagrams and more on how it feels in practice — to the person wearing the watch, and to everyone else in the room who didn't get a chime.