The letter said the Port Authority is exploring the idea of discontinuing service with the TSA in favor of a private security screening company.
It isn't the first threatening letter that the TSA has received in recent months. Atlanta's international airport may also be shopping around after becoming frustrated with wait times.
Twenty-two airports across the nation use private screening companies, including some major airports like San Francisco International Airport and Kansas City International Airport.
But for those that still use the TSA, traveler frustration is growing. A new social media campaign calls for travelers to post pictures of long security lines at airports across the nation with the hashtag #IHateTheWait.
The TSA said it's taking several steps to speed things up, including hiring more staff and encouraging pre-checks.