|How do I accurately calculate code coverage prior to a deployment in a destination org?
1. In the destination org, clear test history (Setup->Apex Test Execution-> click link 'View Test History'->click button 'Clear Test Data')
NOTE: The 'Clear Test Data' button completes the following 2 actions:
ì) Deletes all entries from the ApexCodeCoverage table
ii) Resets all the entries in the ApexCodeCoverageAggregate table to have 0 lines covered and all lines marked as uncovered
This will result in seeing 0% code coverage in the Organizations estimation
2. As the above action only clears history from the 'ApexCodeCoverage' table, we also need to manually clear entries for the 'ApexCodeCoverageAggregate' table to ensure there are no corrupt/invalid counts in the lines per class. Via the Developer Console, execute the following query under the 'Query Editor' tab, using the 'Tooling API':
SELECT Id FROM ApexCodeCoverageAggregate
3. Highlight every row returned and click the button 'Delete Row'
**NOTE: This process is asynchronous. Keep checking this by pressing the 'Refresh Grid' button in the Developer console until all rows are cleared.
4. Ensure there are no records in the 'ApexCodeCoverageAggregate' table by re-running the same query above in step 2 or hitting by the 'Refresh Grid' button as mentioned in the note above.
5. Now we have fully cleared out all test history, all Apex classes should be recompiled via 'Compile All Classes' link in UI (Setup->Develop->Apex Classes->click link 'Compile all classes')
6. Once all classes have been recompiled, all tests should be run again via the 'Run All Tests' button in UI (Setup->Develop->Apex Classes->click button 'Run All Tests')
NOTE: When the "Run All Tests" button is clicked in the UI, All Apex tests in the current org will be run. The button does not take into consideration the filter (View) applied on the page.
7. Once all tests are complete, perform an estimate of the destination org's code coverage via the 'Estimate your organization's code coverage' link in UI (Setup->Develop->Apex Classes->click link 'Estimate your organization's code coverage')
8. Take note of the estimated results
9. Verify these results via the ''ApexOrgWideCoverage' table, again via the Developer Console, execute the following query under the 'Query Editor' tab, using the 'Tooling API':
SELECT PercentCovered FROM ApexOrgWideCoverage
10. Again take note of the coverage, hopefully it should match the estimate we received on step 7
11. We can further verify the accuracy of these code coverage results by performing a "dummy" deployment validation, in a source sandbox org, create a simple change set containing no Apex code. This "dummy" change set must contain a component to force a run of all tests as per the documentation (https://www.salesforce.com/us/developer/docs/api_meta/Content/meta_deploy_running_tests.htm), for example a custom field
12. Once this change set containing no Apex code has been uploaded as an outbound change set in the source org, we should perform a 'Validate' of this change set in the destination org under the inbound change sets section
13. Observe the outcome of the validation, as we are validating a change set with no new or modified Apex code, the code coverage calculated should only come from what already exists in destination org's code base. The expectation on the outcome of the validation of this "dummy" change set would be for the results to match the results observed on Step 8 & 10 above. If the results were < 75%, we can expect this change set to fail validation for a code coverage violation, if > 75% then we would expect it to succeed. As mentioned previously, as this change set is not deploying any new/modified code, only the coverage from the destination org's existing code base should be considered
14. Based on the above steps, we should now have an accurate picture of the current state of code coverage in the destination org
How will new code being deployed impact code coverage on deployment?
1. On the deployment of new Apex code, the coverage is calculated using the coverage from the destination org's existing code base (as verified in the above steps) as well as the coverage of the new code being deployed. Should we be deploying new significantly covered code, we can expect an increase to the overall coverage over what we observed above in our change set containing no Apex code, however the overall increase observed may be less than we expected depending to what the this new significantly covered code is relative to.
If previously the first change set containing no Apex code was failing due to low coverage, we may still experience a failing change set again on the second attempt containing new code, however we should see an increase in the coverage, how much is dependent upon the size of the new class and % covered. If an increase is observed as described, then it would certainly appear everything is in order and simply more coverage needs to be written in the source org and deployed in the change set to get it over the 75% threshold line.
For example, if we are deploying a brand new Apex class with 95% coverage with 100 lines newly covered to a destination org with 70% coverage with 10,000 lines covered (can be determined in the above steps 1-14), the following overall code coverage on deployment of this new class can be calculated as follows:
n1 is the existing number of lines to cover,
n2 is the existing number of lines covered,
n3 is the net new number of lines that need coverage, and
n4 is the net new number of lines covered,
then we know n2/n1 will be 70%, and n4/n3 will be 95%, but the new code coverage will be (n2+n4)/(n1+n3) “total lines covered (new and existing) divided by the total number of lines to cover (new and existing)”
So if n1=10,000, n2 = 7,000, n3 = 100, and n4 = 95, new code coverage will be (95+7000)/(10000+100) = 70.25%
2. This can once again be tested by performing a validation of a change set containing the new code. Assuming we have already performed our steps and analysis on the destination org's existing code base in steps 1-14, in a source sandbox org, create a simple change set only containing this new Apex class which is 95% covered with 100 lines covered
3. Once this change set has been uploaded as an outbound change set in the source org, we should perform a 'Validate' of this change set in the destination org under the inbound change sets section
4. Taking our example previously, if we only had 70% coverage in the destination org's existing code base, the deployment of this new class should increase the overall coverage to 70.25%, but the deployment would fail due to code coverage violation. As mentioned previously, we would need write further code coverage and include this in the deployment change set to get this deployment over the 75% threshold line, perhaps including new test methods to cover uncovered code in the existing code base as the 70% observed there is too low
Additional code coverage considerations when troubleshooting
- Ensure all tests pass and at least 1% of coverage is applied to all triggers
- Ensure test classes are not being considered in the overall coverage calculation. See Why is a Test class evaluated as part of the Organization’s Code Coverage? for more information
- If we see a decrease in coverage in the destination org since last deployment, follow the steps detailed above and clear history in full. Review all test classes for use of @isTest(SeeAllData=true) as a data or metadata dependency may be responsible
- Identify any test classes using API version 23.0 or lower, as the default for these classes is to use @isTest(SeeAllData=true)