Apple starts testing watch software that listens to and summarizes what people around you say

Apple has pushed its new “Audio Intelligence” features into beta testing as part of iOS 27.3, starting Tuesday. The update brings four opt-in AI tools to the Apple Watch Series 12 and Apple Watch Ultra 4, including two that actively listen to conversations.

The features were first shown off at Apple’s Surprise and Shine hardware event in September. The beta rolled out to developers first, announced through a press release sent by Apple in India. As of Tuesday morning, it had not yet reached the U.S. market.

What the four features actually do

Two of the features are fairly uncontroversial. Sound recognition alerts deaf and hard-of-hearing users to sirens, alarms, or doorbells through the watch. Song recognition uses Apple’s Shazam technology to identify music playing nearby.

The other two, Live Rewind and Siri Recaps, are built around listening to human speech. Live Rewind lets you double-press the digital crown to see a text snippet of whatever was said in the last 15 seconds, useful if someone rattles off directions or a restaurant lists its specials too fast to catch. You can let the snippet vanish or save it to the Siri app. An “Ask Siri” option also appears below the text, so you can ask about an unfamiliar term or detail without interrupting the person talking.

How Apple is trying to avoid a privacy backlash

Apple built in visible and audible signals for a reason. Double-pressing the crown triggers a chime and an on-screen animation, a deliberate cue meant to show bystanders that something was just recorded. Unsaved snippets disappear automatically after 30 seconds.

Siri Recaps goes further, generating short summaries of entire conversations with bullet points covering things like deadlines or pricing. Apple says it does not identify speakers, store raw audio, or produce word-for-word transcripts. The company suggests use cases like meetings, parent-teacher conferences, or training sessions, essentially replacing the notes you’d otherwise take yourself. It won’t run while you’re using headphones with music or noise cancellation on.

Users get a daily prompt to review and decide which Siri Recaps to keep, and unsaved recaps delete themselves within seven days. The review reminder also shows up each morning when you unlock the watch. Recaps can be scheduled, say, only during work hours or never at night, and the feature can be switched off instantly from Control Center. Apple says sensitive details like government IDs, financial data, and language related to self-harm are filtered out before a recap is generated.

The hardware doing the heavy lifting

None of this works without new silicon. The features depend on the Watch’s S11 chip, its built-in microphone, Apple Intelligence models running on the paired iPhone, and Apple’s Private Cloud Compute system.

The always-on processor includes a secure enclave, a hardware-protected memory space the rest of the operating system can’t touch. A dedicated audio buffer inside that enclave processes raw audio so it’s never actually accessible to anyone, including Apple itself. That architecture is clearly a response to the backlash aimed at other AI wearables, like Meta’s AI glasses, where ambient recording raised concerns among bystanders who never agreed to be listened to.

Whether that’s enough to satisfy critics is a separate question. A chime and an animation tell the person wearing the watch that something was recorded, but they don’t necessarily tell everyone else in the room.

Worth Watching

  • When will Audio Intelligence actually reach U.S. users, and will the rollout differ by region?
  • Will the audible chime be enough to reassure people who don’t want their words recorded, even briefly?
  • How will Apple’s filtering of sensitive information hold up once the feature sees real-world use at scale?

Source: TechCrunch

Leave a Reply

Your email address will not be published. Required fields are marked *

Verified by MonsterInsights