NLTK flaw lets crafted regex patterns tie up CPU during compilation

  • NLTK
  • denial of service
  • regex

Vulnerability affects versions 3.10.0 through 3.10.3

Crafted patterns can exhaust CPU resources via compilation

Existing match-only timeouts do not cover compilation phase

NLTK has a compile-time regular-expression denial-of-service vulnerability in versions 3.10.0 through 3.10.3, according to the security advisory. The flaw can let an attacker tie up CPU resources before a search or tokenization operation begins.

The issue arises when an application passes untrusted text as a pattern to NLTK’s TokenSearcher.findall(), Text.findall() or RegexpTokenizer. The third-party regex engine’s compilation cost grows roughly quadratically with the number of capturing groups.

The advisory says a pattern of about 50 KB made up of empty groups can keep a CPU worker busy for tens of seconds. NLTK’s timeout applies to matching, not compilation, so it does not prevent this CPU consumption.

Key terms explained

Denial-of-service

A denial-of-service attack prevents legitimate users from accessing network resources or services. This is often achieved by overwhelming a system with traffic until it crashes or becomes unresponsive.

Think of it this way. It is like a crowd of people blocking the entrance to a store so that actual customers cannot get inside to shop.

NLTK

NLTK is an open-source platform used for developing Python programs that process human language data. It provides libraries for tasks such as tokenization, classification, and linguistic analysis.