One of the great promises of artificial intelligence was that it would help to cure diseases that have thus far stumped humanity. It hasn’t gotten there yet, but it has now created a new, viable virus not found in natureâwhich is one of those technical achievements that is both impressive and unnerving.

Researchers at the Palo Alto, California-based Arc Institute believe they have accomplished a new first by using AI to manufacture new viral genomes by first teaching a model how to recognize the DNA structure of different viruses found in nature and then tasking it to create recipes for new, novel viruses.

Once the modelâa genome language model called Evoâwhipped up some options, the scientists created those molecules and loaded them up with bacteria to see what would happen. According to the researchers, who detailed the experiment in a study published Thursday in Science, the bacteria produced viruses capable of infecting other bacteria and spreading.

This experiment comes with a couple of anti-doomer caveats. The first is that the process of synthesizing viruses existed long before AI showed up on the scene, so part of this experiment is just an existing tool in the toolkit of viral researchers. The second is that this particular virus is not a threat to humans and can only infect bacteria. That’s by design, as the DNA the AI model was pulling from was a relatively harmless virus known as Phi X-174.

There’s also the fact that the AI really took a lot of cracks at it to achieve its goal. According to The New York Times, the researchers had Evo generate 700,000 potential genomes, then narrowed it to about 300 that they made in the lab. Just 16 of them proved to be viable. It’s a far cry from the Infinite Monkey Theorem, but it is a pretty low hit rate.

But that’s probably not enough to provide peace of mind for anyone worried about how this technology might be utilized. The model needed hundreds of thousands of attempts to produce a handful of viable viruses, but those that worked were reportedly able to overcome resistance in two strains of E. coli, suggesting they were more than crude imitations of naturally occurring viruses.

Science ran an accompanying article alongside the study authored by Professor Tom Inglesby and Dr. Moritz Hanke at the Center for Health Security at Johns Hopkins University, issuing a warning about the breakthrough. âAlthough this is promising for life sciences applications, it also raises urgent biosafety and biosecurity questions,” they wrote. “The ability to compose viral genomes using generative AI now exists; the governance to safely steer it does not.â

To that end, the National Institutes of Health issued new policies last week for preventing high-risk life science research by barring scientists from conducting experiments that would make biological agents more dangerous. What it doesn’t do, though, is restrict AI-based research. “Purely computational (i.e., in silico) research that may include development of computational models and software, or may use such means to design novel forms of biological agents, is not prohibited by this policy unless it involves an entity of concern,” the document reads.

An entity of concern implies something that already existsâthe Times raises smallpox as an obvious example. But what if they whip up a new harmful virus? How do the rules apply then? Just another nightmare scenario over which you have no personal control to keep you up at night.