Skip to content
AI Primer
breaking

Developers report Firebase payload crashed iOS apps

Developers traced widespread iOS app crashes to a malformed server-side Firebase payload, which was rolled back. Cached payloads reportedly kept some apps failing afterward.

3 min read
Developers report Firebase payload crashed iOS apps
Developers report Firebase payload crashed iOS apps

TL;DR

One developer's minimal reproduction crashed on three of three fresh installs after receiving a 225-byte sdk-exp response with a zero-length flag. An offline parser test in the same thread found that the SDK could record a successful fetch before converting that response into experiment objects.

A zero-length flag in `sdk-exp`

The original GitHub report traced 42 of 42 inspected crashes to an HTTP 200 response from POST https://app-analytics-services.com/sdk-exp. Its stack ended in NSInvalidArgumentException: key cannot be nil on the Analytics experiment worker queue.

  • A captured, gzipped protobuf response contained an empty flag entry, encoded as 12 00, among flags with names and values.
  • The SDK converted the unnamed flag into a nil key for GULMutableDictionary, which then tried to insert it into an Objective-C dictionary.
  • The dictionary write ran asynchronously, leaving the crashing thread without the parser's caller frames in its stack trace.

Existing builds and named apps

The issue's developer reports identified affected Firebase SDK versions including 11.11.0, 12.4.0, 12.8.0, 12.17.0 and 12.19.2. The initial reporter saw the crash begin simultaneously on four released builds without shipping an update. A DoorDash developer reported thousands of crashes across its apps; Quizlet also reported the issue.

Rollback at 19:52 PDT

Google software engineer Nick Cooke's published incident timeline put the start at 17:41 PDT on September 28 and the completed server-side fix at 19:52 PDT, two hours and 11 minutes later. He said no SDK update was required.

The four-hour fetch clock

Google said some instances could continue crashing until 23:52 PDT, four hours after the completed fix, because of caching, according to Cooke's timeline. The minimal reproduction exposed another detail: its app wrote a successful-fetch timestamp just before parsing crashed, then reopened without immediately fetching again.

An offline SDK test in the issue identified a 14,400-second experiment fetch interval. That helps explain why the reproduced crash happened roughly once per fetch interval rather than on every relaunch, though the tester had not confirmed timestamp persistence on affected devices.

GitHub replies and the status dashboard

The GitHub issue opened at 00:59 UTC on September 29, 18 minutes after the reported start. The GitHub thread update said a Firebase developer responded after roughly 100 messages; the status-page complaint said the Firebase dashboard showed no incident even five hours in.

The Firebase Status Dashboard directs Google Analytics incidents to the separate Ads Status Dashboard. Google's published timeline identifies the malformed payload but gives no account of how it reached production; at the time of the postmortem request, no public postmortem had been promised.

Further reading

Discussion across the web

Where this story is being discussed, in original context.

On X· 3 threads
TL;DR3 posts
Existing builds and named apps1 post
GitHub replies and the status dashboard3 posts
Share on X