Problem

How do I reject incoming email for unknown users in MS Exchange 2013?
How do I setup a recipient filter to reject unknown users in MS Exchange 2013?

Solution

Recipient filters work differently in Microsoft Exchange 2013 compared to previous version. As you can see from the session below the reject is done after the DATA is processed.

mail from:<This email address is being protected from spambots. You need JavaScript enabled to view it.>
250 2.1.0 Sender OK
rcpt to:<This email address is being protected from spambots. You need JavaScript enabled to view it.>
250 2.1.5 Recipient OK
data
354 Start mail input; end with 
Write some Text Here.
.
550 5.1.1 User unknown

The below example shows you how to setup a recipient filter for Microsoft Exchange to reject unknown users.

This solution is for Exchange Servers 2013 and using the anti-spam agent for exchange. This is an available feature in the Edge Transport Server. However if you don't have Edge Transport Server it is possible to install the anti-spam agent in the Hub Transport Server. Instructions for this can be found here.

First lets verify the Anti-Spam Functionality is enabled

1. Start the Exchange Management Shell (EMS)
2. Type the following:
   
   Get-TransportAgent
   
   Make sure the recipient filter agent is available and enabled. If its not available then follow the link above on installing the anti-spam agent. If is available but disabled you can enable it by the below command
   
   Enable-TransportAgent "Recipient Filter Agent"

Restart the Exchange Transport service after making the above changes.

The next step is to ensure your accepted domains are using the AddressBook for checking for valid recipients. By default this should be enabled when you set up Exchange as an authoritative Mailbox Server for you domain.

3. To check your server is going the AddressBook for validation do the following
   
   Get-AcceptedDomain | Format-List Name,AddressBookEnabled
   
   It should provide you with a list of all accepted domains and if the AddressBook is enable or not. If by any chance Exchange is not Authoritative and the AddressBook is disabled then enable it with:
   
   Set-AcceptedDomain <name of accepted domain> -AddressBookEnabled $true
   
   Or, to enable for all domains (caution, make sure you are not relaying any domains before runing this)
   
   Get-AcceptedDomain | Set-AcceptedDomain -AddressBookEnabled $true

Now you should have Recipient Filter enabled on you Mailbox Server and AddressBook enabled on you domain. But, if you test this now, it probably still won't work. That's because Validation is still disabled.

4. To check the status of validation run the following
   
   Get-RecipientFilterConfig | FL Enabled,RecipientValidationEnabled
   
   It should return that Recipient Filter is enable, but validation is not
   
   

   Enabled : True
   RecipientValidationEnabled : False

5. To enable validation run the following
   
   Set-RecipientFilterConfig -RecipientValidationEnabled $true
   
   
6. Restart the Exchange Transport service

If we now test we should see a reject on the unknown user.

mail from:<This email address is being protected from spambots. You need JavaScript enabled to view it.>
250 2.1.0 Sender OK
rcpt to:<This email address is being protected from spambots. You need JavaScript enabled to view it.>
250 2.1.5 Recipient OK
rcpt to:<This email address is being protected from spambots. You need JavaScript enabled to view it.>
550 5.1.1 Address Unknown