From OpenSSL Zero-Days to HAWK: How AI Is Changing Cryptanalysis

During the first seven months of 2026, artificial intelligence crossed an important boundary in cryptographic security.

AI systems were already useful for reviewing source code, generating fuzzing harnesses, and helping security researchers investigate crashes. Recent events show something broader. Agentic systems are now finding significant vulnerabilities in heavily tested cryptographic libraries, constructing working exploits, proposing patches, and contributing original mathematical attacks against cryptographic designs.

The most visible examples involve OpenSSL, wolfSSL, the withdrawn HAWK post-quantum signature candidate, and a new attack against reduced-round AES.

These events do not mean that modern encryption has suddenly been broken. They do show that the economics and speed of cryptanalysis are changing.

Two Different Forms of AI Cryptanalysis

The recent results fall into two categories that should not be confused.

Implementation vulnerability discovery

In this category, the cryptographic algorithm may remain mathematically sound, but the software implementing it contains a defect.

Examples include:

  • Failure to verify an authentication tag
  • Incorrect certificate-chain validation
  • Buffer overflows in ASN.1, CMS, or TLS processing
  • Nonce or initialization-vector reuse
  • Weak or missing parameter validation
  • Timing leakage or error-based padding oracles

These flaws can defeat security even when the underlying algorithm is secure.

Mathematical cryptanalysis

Mathematical cryptanalysis attacks the design or its security assumptions rather than a particular software implementation.

This work may involve:

  • Reducing a problem to a lower-dimensional lattice
  • Improving a meet-in-the-middle attack
  • Finding an invariant or previously unnoticed symmetry
  • Identifying a weakness in a security proof
  • Reducing the time, memory, or data complexity of key recovery

The HAWK and reduced-round AES results are important because they move beyond automated code auditing and into original mathematical research.

OpenSSL: Autonomous Systems Find Bugs in a Mature Codebase

OpenSSL is one of the most reviewed and widely deployed cryptographic libraries in the world. That makes the number of recent AI-assisted findings especially notable.

AISLE reported that its autonomous analysis system discovered all 12 vulnerabilities included in OpenSSL’s coordinated January 27, 2026 security release. OpenSSL’s official vulnerability records credit AISLE researchers for the findings, including CVE-2025-15467, a high-severity stack buffer overflow in CMS AuthEnvelopedData processing that could cause a crash or potentially permit remote code execution.

The January findings also covered:

  • PKCS#12 parsing errors
  • Out-of-bounds writes
  • TLS and QUIC denial-of-service conditions
  • Authentication and integrity failures
  • Incorrect handling of large inputs

On April 7, OpenSSL published another advisory containing seven vulnerabilities. AISLE stated that its system found five of them. One issue, CVE-2026-28386, was independently reported by both AISLE and Anthropic. It involved an out-of-bounds read in the AES-CFB-128 assembly path on x86-64 systems with AVX-512 and VAES support.

The pattern continued in June. OpenSSL’s June 9 vulnerability release credited Anthropic researchers on multiple findings, including:

  • CVE-2026-34181: PKCS#12 files using PBMAC1 could be accepted with extremely short HMAC keys.
  • CVE-2026-34182: CMS AuthEnvelopedData processing could accept forged or insufficiently authenticated messages.
  • CVE-2026-42768: A multi-recipient CMS and PKCS#7 path could expose a Bleichenbacher-style RSA oracle.
  • CVE-2026-45445: The AES-OCB one-shot API ignored the caller’s IV, causing effective nonce reuse and potentially universal ciphertext forgery.
  • CVE-2026-45446: AES-SIV and AES-GCM-SIV mishandled authentication for empty messages.

The significance is not simply the number of CVEs. These systems are finding semantic cryptographic failures that traditional pattern matching can miss. An implementation can compile successfully, pass ordinary tests, and still violate the security contract of an authenticated-encryption mode, certificate validator, or key-transport mechanism.

wolfSSL: Eight AI-Generated CVEs Trigger a Release

wolfSSL reported a similarly high-profile result in April 2026.

The company gave Anthropic’s Claude Mythos Preview access to its approximately two-million-line codebase after existing static analysis, fuzzing, internal AI tools, peer review, and compliance audits had already been applied. According to wolfSSL, the resulting findings generated eight CVEs and triggered the release of wolfSSL 5.9.1.

The broader 5.9.1 release contained 22 CVEs, including one critical and ten high-severity issues. The AI-assisted findings included several serious cryptographic failures:

  • CVE-2026-5194: Missing digest-size and algorithm-identifier checks weakened signature verification across configurations involving ECDSA, DSA, ML-DSA, Ed25519, and Ed448.
  • CVE-2026-5466: The ECCSI verification path could accept a crafted forged signature.
  • CVE-2026-5477: AES-EAX and CMAC authentication could be bypassed for messages larger than 4 GiB.
  • CVE-2026-5500: A PKCS#7 AES-GCM tag could be truncated to one byte, reducing authentication strength from 128 bits to 8 bits.
  • CVE-2026-5501: The OpenSSL-compatibility certificate-verification API could accept a forged leaf certificate under specific chain conditions.
  • CVE-2026-5503: Encrypted ClientHello SNI handling could produce a heap overflow.
  • CVE-2026-5479: The EVP-compatible ChaCha20-Poly1305 decryption path could return plaintext without verifying the authentication tag.

These were not all simple memory-safety bugs. Several represented failures to enforce the intended mathematical security properties of cryptographic protocols.

wolfSSL’s response is also instructive. Rather than treating the volume of findings as evidence that AI reports should be ignored, the company added further AI-based testing and publicly argued that maintainers are entering a period of higher-volume, higher-quality vulnerability reporting.

HAWK: AI-Assisted Mathematics Changes a NIST Competition

The most consequential mathematical result involved HAWK, a lattice-based digital signature algorithm that NIST advanced to Round 3 of its Additional Digital Signatures process in May 2026.

Researchers using Claude Mythos Preview discovered an improved key-recovery attack that reduced recovery of an equivalent HAWK secret key to a Shortest Vector Problem in dimension approximately:

n / 2 + 1

The resulting estimates lowered the gate-count attack cost:

Parameter set Earlier estimate Revised estimate
HAWK-512 2^150 2^108
HAWK-1024 2^288 2^182

The researchers also demonstrated an end-to-end recovery against the smaller HAWK-256 challenge parameters in a few hours on one server.

Anthropic described the discovery as semi-autonomous. A multi-agent workflow reviewed prior research, generated hypotheses, ran mathematical and computational experiments, and produced a verification pipeline. The work reportedly required about 60 hours and approximately $100,000 in model usage.

The HAWK team confirmed that the attack approximately halved the lattice-reduction block size needed for key recovery. It concluded that straightforward mitigations—such as doubling parameters or moving to higher-rank modules—would make HAWK uncompetitive.

On July 29, the team withdrew HAWK from NIST’s standardization process, and NIST updated the official Round 3 page to record the withdrawal.

This does not affect ML-DSA, FN-DSA, ML-KEM, or lattice cryptography generally. The attack was specific to HAWK. Its importance lies in the process: a candidate that survived years of human analysis was materially weakened by an AI-assisted result shortly after entering the final evaluation round.

Reduced-Round AES: A More Autonomous Research Result

Anthropic also reported a new attack against seven-round AES-128.

Full AES-128 uses ten rounds. Cryptographers analyze reduced-round versions to measure security margins and develop techniques that might eventually generalize.

In this case, an autonomous agent improved the previous best meet-in-the-middle approach by developing a fingerprinting technique called a Möbius Bridge. The technique removed one 256-way guess from the attack and, after accounting for computational overhead, reportedly improved the attack by approximately 200 to 800 times.

The model generated hypotheses, tested them computationally, rejected failed approaches, and refined the final attack after producing roughly one billion output tokens. Human researchers then spent hundreds of hours validating the result.

The attack does not break full AES-128 and has no immediate impact on production systems. Its significance is methodological. Unlike the HAWK work, which included occasional human steering, the central reduced-round AES result was reportedly discovered almost entirely autonomously inside a research scaffold.

Anthropic has also described preliminary reduced-round results involving LEA, Serpent-128, Salsa20, Poseidon, and SHA-1. Those claims should be treated more cautiously until complete methods and independent reviews are available.

CryptanalysisBench Turns Capability Into a Measurable Trend

A new benchmark called CryptanalysisBench provides broader evidence that these results are not isolated demonstrations.

The benchmark contains 191 tasks across six families of cryptographic primitives, with many drawn from four NIST standardization competitions. It includes:

  1. Schemes with known practical breaks
  2. Full-strength and reduced variants without known practical breaks
  3. Frontier challenge problems involving production-oriented primitives

According to the initial paper, five frontier models solved between 65% and 86% of the first-tier schemes. They also produced novel results, including a key-recovery attack against the SpoC authenticated-encryption design and identification of an error in KINDI’s published CCA-security proof.

Benchmarks cannot establish that a model is a reliable autonomous cryptographer. They can, however, track whether models are progressing from reproducing known attacks to finding previously unpublished ones.

What Has Actually Changed?

The core change is not that AI has replaced cryptographers.

It is that a cryptographer—or a security team—can now launch many parallel agents that:

  • Read specifications and source code
  • Search prior literature
  • Generate attack hypotheses
  • Build test harnesses
  • Run symbolic and numerical experiments
  • Construct proof-of-concept exploits
  • Critique competing hypotheses
  • Draft patches and technical reports

This turns cryptanalysis from a mostly serial human activity into a partially parallel machine-assisted process.

The models still fail frequently. Many sessions produce no useful result, and plausible mathematical arguments can be wrong. Human researchers remain essential for validation, responsible disclosure, severity assessment, and determining whether an attack affects a real deployment. Anthropic itself reported extensive human validation for both the software vulnerabilities and mathematical results.

The practical shift is that the cost of exploring a large number of ideas is falling rapidly.

Implications for Cryptographic Engineering

Continuous AI-assisted review should become normal

A one-time audit is no longer sufficient for important cryptographic software. Security teams should combine AI analysis with fuzzing, sanitizers, static analysis, formal methods, differential testing, and expert review.

Validation must remain evidence-based

AI-generated findings should include reproducible test cases, attack code, machine-checkable proofs where possible, affected-version analysis, and clear threat models. Unsupported reports can overwhelm maintainers just as easily as good systems can help them.

FIPS status does not eliminate implementation risk

A validated cryptographic module covers a defined boundary and configuration. Parsing, certificate processing, protocol logic, compatibility APIs, and application integration may sit outside that boundary.

The recent OpenSSL disclosures include examples both inside and outside FIPS module boundaries. Organizations must therefore evaluate the entire cryptographic service, not merely the existence of a validation certificate.

Cryptographic agility becomes more urgent

The HAWK withdrawal shows how quickly an algorithm’s status can change when a new attack appears. Systems should be designed to replace algorithms, parameter sets, certificates, and protocol combinations without a complete redesign.

Cryptographic inventories need implementation detail

A useful cryptographic bill of materials should record more than algorithm names. It should identify:

  • Library and version
  • Enabled build options
  • Hardware-specific code paths
  • Protocol and API usage
  • Certificate-validation behavior
  • FIPS module boundary and certificate
  • Experimental and post-quantum features
  • Upgrade and replacement dependencies

Many recent vulnerabilities affected narrow configurations or compatibility APIs. Without this detail, an organization cannot determine whether it is exposed.

Conclusion

The last several months have produced the clearest evidence yet that AI is becoming a meaningful force in cryptanalysis.

In OpenSSL and wolfSSL, AI systems found exploitable implementation defects that survived mature testing programs and years of review. In HAWK, an AI-assisted mathematical attack directly contributed to the withdrawal of a NIST post-quantum signature candidate. In reduced-round AES, a more autonomous workflow produced a new attack against one of the most studied cryptographic designs in history.

None of these events means that full AES, NIST’s approved post-quantum standards, or modern cryptography as a whole have been broken.

They do mean that the review cycle is accelerating.

The organizations best prepared for this shift will not be those that assume their current algorithms and libraries are permanently safe. They will be those that continuously test implementations, demand reproducible evidence, maintain accurate cryptographic inventories, and build systems capable of changing when the next result arrives.