Technology behemoths are clearly not reading the room, or they are just moving faster than they should when it comes to AI deployment. Google has already stuffed its Gemini AI in every corner of its software stack, from Android to daily productivity tools like Gmail that are used by hundreds of millions of users every day. AI is everywhere. Not all of it is bad, mind you. But in a few places, it just feels forced.

In its latest AI-fication experiment, the company targeted Google Earth. The idea was to let the audience use its Nano Banana 2 AI image generator and make images that can be placed on the map view. On paper, it’s a cool idea. What would the Colosseum of Rome look like in your urban neighborhood? Yeah, fun stuff like that.

What if I put a nuclear reactor on the map of an enemy state, or a tank on the soil of a war-torn nation? Well, that’s a problem, and this is exactly what Google Earth produced without any inhibition.

What’s the big fuss, huh?

Alright, let’s go back to the origins and the good intentions first. This is what Google had in mind:

“For the first time, you can generate custom images using Google Earth’s satellite, aerial, and 3D imagery alongside Nano Banana, which creates concepts grounded in the real world. Just zoom in to a place in Google Earth on web, tap “create image,” and type whatever you want to see.”

Here’s the problem. AI is not always a welcome guest.

In fact, Google’s AI has been misused in numerous ways by multiple bad actors in the not-too-distant past. In a disclosure submitted before Australia’s eSafety Commission, it was reported that Gemini AI was abused to create AI-generated deepfake terrorism and child abuse material. Earlier this year, Google’s Cloud unit disclosed usage of Gemini by cybercriminals engaged in malware and surveillance operations.

Merely a few days ago, Google filed a lawsuit against a China-linked network over usage of Gemini to launch large-scale phishing campaigns. Even for a regular user, it’s easy to trick Google’s AI image generator into creating something as disturbing as a Nazi Spongebob. The situation with OpenAI’s ChatGPT image generation model is not too different.

Google just flew dangerously close to the proverbial Sun. The latest AI playground for the company was Google Earth, and the misadventure lasted merely a day. Henk van Ess, an online research expert, gave the new AI feature a spin and quickly tore it to shreds. This is what he wrote in his newsletter, Digital Digging:

“Tonight I typed just one sentence into Google Earth and put refugees near the Mexican border. Then I planted a nuclear plant in Iran. Then I put a fatal crash on a street in Amsterdam. Google’s own satellite imagery underneath all three. What on earth is Google doing?”

So many layers

That was not the end of the problem. The Nano Banana engine simply complied with the problematic AI image generation requests without any counter. “Nothing was refused, nothing was softened, and nothing suggested I try a different prompt,” van Ess wrote.

Let’s give some quarter to Google here. AI image generators usually focus on avoiding stuff like sexualized imagery, weapons, racist stereotyping, and other such obviously bad requests. It can’t handle stuff that seems normal, but can stir an international relations storm. Placing a nuclear reactor on the map of Iran, or putting a bomb crater on a Gaza hospital, are two such samples of “innocent failures,” if I can put it that way.

But that’s not the end of Google’s AI oversight. The AI-generated imagery on Google Earth passed through AI checkers without raising any alarms. BBC also stress-tested the tool, and it coughed up controversial stuff such as Russian tanks moving on Ukrainian land, the Eiffel Tower lying collapsed on the ground, and the great pyramids of Egypt swallowed by a giant sinkhole.

Worryingly, the research notes that it was possible to trick Google’s Gemini into saying that these fake images were legitimate. Unsurprisingly, third-party AI checker tools also failed in numerous instances. To recall, all AI-generated images created by Gemini (and Nano Banana) come with an invisible SynthID watermark (developed by Google DeepMind) and a visible watermark, as well.

That’s not even my biggest concern. The internet is brimming with eerily realistic AI-generated content, and a lot of people don’t even suspect that it’s fake. And even if they suspect, they simply don’t know how to verify its authenticity and detect if it was AI. In a lot of cases, people simply don’t have the incentive to fact-check doubtful images.

“Journalists might be able to check that image… and report on it. But again, not everyone is going to do that,” AI researcher, Henry Ajder, told the BBC. I talked to three fact-checkers associated with the Poynter-backed International Fact-Checking Network (IFCN), and they told me that there is a huge digital literacy gap. And yeah, in most cases, they simply don’t have a reason to verify a synthetic image’s true origin and flag it publicly.

This was just bad

Nonetheless, Google’s AI experiment with letting users express their creativity was a case of stunning ignorance. Thankfully, the company did a course correction and pulled it quickly.

“We’ve seen geospatial professionals using this feature for a range of useful purposes, however we’ve also seen people sharing screenshots of generated imagery that appear to violate our policies. So we’re rolling back this feature in Google Earth while we work on implementing stronger guardrails,” the company shared in a statement on its official blog.

I suspect that this feature rollback was swift because the stakes were much higher. AI image models can easily be tricked into violating copyright guardrails, produce sexualized images, and raise all kinds of visual hell. Look no further than the Grok AI on X, formerly Twitter. Google pulled this feature so urgently, I reckon, because it could escalate into an international storm.

Google bows down to political powers and shows a different territorial map, depending on where you live. The border disputes between India-Pakistan and India-China are two such examples, and you can read more about them in this analysis by The Washington Post. Also, let’s not forget the incredible international pressure on Google when it comes to AI supremacy wars and the data center protests.

A nuclear reactor in Iran during times of active conflict with the US? Well, I’m not surprised (and also glad) that the tech giant quickly course-corrected on its Google Earth-AI flub. Hey Google, maybe, stick to fun AI experiments such as using Nano Banana to digitize my wardrobe and level up my fashion game?