Customers can sometimes see a difference between the number of Apex Test failures when running all tests or a single test via the UI compared to the failure count observed when performing a deployment. A comparison between the differences leads to the conclusion that the differences can be accounted for by observing 'Connection Canceled' errors in the UI but not in the deployment, why is this?
During deployments, we do not do fixed test run time limit checks, whereas via the UI we do (either 'Run All Tests' or running a test for an individual class). In the deployment context, this limit is a bit relaxed. If the differences between the test failures seen in the UI compared to the deployment are solely down to 'Connection Canceled' errors, then this can be deemed expected behavior as we intentionally do not do this hard check in a deployment.
NOTE: Same behavior can also be observed when simply running tests outside of a deployment/validation scenario via the Metadata API, again this is expected behavior
000396102

We use three kinds of cookies on our websites: required, functional, and advertising. You can choose whether functional and advertising cookies apply. Click on the different cookie categories to find out more about each category and to change the default settings.
Privacy Statement
Required cookies are necessary for basic website functionality. Some examples include: session cookies needed to transmit the website, authentication cookies, and security cookies.
Functional cookies enhance functions, performance, and services on the website. Some examples include: cookies used to analyze site traffic, cookies used for market research, and cookies used to display advertising that is not directed to a particular individual.
Advertising cookies track activity across websites in order to understand a viewer’s interests, and direct them specific marketing. Some examples include: cookies used for remarketing, or interest-based advertising.