Free AI For Everyone, What Could Possibly Go Wrong? (Don't Worry, Biotech Terrorists Suck)

Scott Alexander examines the debate over open-weights AI, where major tech companies advocate for freely downloadable models while critics worry about misuse for hacking, bioterrorism, and loss of control. He argues that superintelligent AI takeover risk justifies preemptive action, but criminal misuse risks like hacking and bioterrorism don't—because governments will react forcefully after the first incident, and bioterrorists have a historically dismal track record. He remains neutral on open weights, expecting they'll prove unsustainable in time without needing an immediate ban.