Navigate to Configuration > Chat > Chat buttons and either click "Create" in order to create a button or click "Edit" in order to edit an existing button.
In the 'Overview' section uncheck the "When no agent is online, offline button is displayed and visitor can use contact form." option as shown in the screenshot below:
In the 'Overview' section uncheck the "When no agent is online, offline button is displayed and visitor can use contact form." option as shown in the screenshot below:
that will ensure, that if no agent is online concerning the department (set in the Advanced Settings > Goes to section) to which the button belongs, the chat button won't appear in your site.