Record number is used as “a record number”. This means a number larger than ever before.

For example:

A record number of elephants died in Sri Lanka in 2019.

KLM welcomed a record number of 35.1 million passengers on board in 2019.

Record numbers is used as “in record numbers”. This means more than ever before.

For example:

New houses are being built in record numbers.