A Department of Defense database containing 1.8 billion scraped internet posts over a span of eight years was left publicly exposed, according to researchers from the cybersecurity firm UpGuard.
Researcher Chris Vickery discovered the trove, first reported by CNN. Vickery and UpGuard have made a name for themselves sniffing out mistakenly publicly exposed databases over the last year including data on 200 million voters, one gigabyte of sensitive files from Viacom and information on 14 million Verizon customers.
“With evidence that the software employed to create these data stores was built and operated by an apparently defunct private-sector government contractor named VendorX, this cloud leak is a striking illustration of just how damaging third-party vendor risk can be, capable of affecting even the highest echelons of the Pentagon,” UpGuard’s Dan O’Sullivan wrote in a blog post.
In June, Vickery found 60,000 sensitive files left publicly exposed by leading U.S. government contractor Booz Allen Hamilton.
Vickery found the exposed Pentagon files in three Amazon Web Services S3 cloud storage buckets configured to allow any AWS user to login and view the contents. The Defense Department secured the data after being notified.
It’s been a frustrating year for Amazon watching Vickery make his discoveries. The company has taken to proactively notifying users about publicly exposed data with a product called Macie.
“Amazon Macie is a security service that uses machine learning to automatically discover, classify, and protect sensitive data in AWS,” the product’s website reads. “Amazon Macie recognizes sensitive data such as personally identifiable information (PII) or intellectual property, and provides you with dashboards and alerts that give visibility into how this data is being accessed or moved.”
Macie was introduced in August 2017 and is already in use by marquee firms like Netflix and Hulu. But it’s an extra cost in terms of money, time and knowledge so Macie has a ways to before it covers anything close to the whole of AWS. Until then, expect more Vickery discoveries to come to light.