By default, access is not granted to a user on the day specified as a card expiration date (the Valid till parameter, see Settings panel of the User object).
If Castle integration module is to grant access on the specified day, set the Include last day checkbox on the Castle server object settings panel and click Apply.