Validation rules work with Inline Editing but does not show error message

When performing an inline editing in Classic, the validation rule will be displayed via a popup window. Therefore, check your popup blocker settings to ensure that you receive the error message.

However, remember that every browser has different setting for blocking or enabling popups. So you'll have to experiment with the settings. But once set properly, a popup will display and list all rows that you attempted to update. The popup will list the records successful updated. It will also display unsuccessful updates along with the validation rule error.  

In Lightning Experience, warning icon is displayed beside the cancel and save buttons. 


See also
Considerations for Editing Inline in a List View in Lightning Experience