Print this page

How to Diagnose Network Issues

Knowledge Article Number 000213798
Description

Problem:

You or your users are either unable to connect to Salesforce or your connection is considerably slower than normal.

 

Description:  

These issues can be some of the most frustrating to resolve because they can bring your team to a halt.  Occasionally they can be caused by doing something seemingly innocent like installing a new package, implementing a new sharing rule, or enabling a feature.  However, by far the most common cause for these issues is the link between your users and the Salesforce application - the Internet Service Provider (ISP) and network.  ISP and networking issues will be the focus of this article, with an end goal of giving you the tools to efficiently troubleshoot such issues in order to drive an expedited resolution.

 

Troubleshooting:

  1. Check Trust
    The first thing you should always do in these cases is to check status.salesforce.com/status. On this page we report instance wide availability and performance issues.  Above the chart we will also occasionally post general status messages if we are engaged with an ISP to work through any networking issues.

     

  2. Ask your colleagues.  
    Check if your entire team is also experiencing the same level of performance and whether they have found any ways to mitigate it.  For example, you always use a hard line connection but you have a colleague who uses WiFi and they are not experiencing the issue.  Also, if your company has multiple offices or remote employees please try and check with them - are they seeing better performance?  Does your team have the same performance experience across all your orgs (including sandboxes)?  If any of these scenarios are applicable to this issue then you are very likely experiencing a networking problem and not a Salesforce specific problem.

 
  1. Run a Ping to Salesforce
    From a Windows based system click on the start button, and type “cmd”in the search bar. The top result should be an item called “cmd” - click on this.  In the resulting console window, type
        ping -n 100 [instance].salesforce.com
    where [instance] is your specific instance e.g. if you are on na5 it would be
        ping -n 100 na5.salesforce.com.
    You will start to see lines of text running across your screen.  Depending on your specific route to Salesforce this could take up to a few minutes to complete.  The output of this result will look something like the ping test to na17.salesforce.com below:

     

    ping.jpg          


    These results show that at that moment I had a very good connection to the specified instance.  Indications of errors would include:

    • Packet loss greater than about 10%  (mine shows “0% loss”)

    • A large range of round trip times. While a difference of about 100ms  between times is pretty common, a situation where you have an average round trip time of 105ms and a Maximum of 500ms would indicate that you may be having some networking issues.  
       

  2. Run a Traceroute to Salesforce
    A traceroute will tell us how your traffic is getting to Salesforce.  It will also help to identify where any issues may be occurring.  To run a traceroute, return to the command prompt and type in
        tracert [instance].salesforce.com
    Below is a traceroute I ran to our North American instance NA17 (tracert na17.salesforce.com):

    Good Trace.JPG


    To read this, start with the first line - this is the initial stop your traffic makes after it leaves your computer.  It will likely be either a 10.X.X.X number or a 192.168.X.X number.  These are reserved for private networks and are also fairly common further down a traceroute.  Further down the route they just indicate that traffic is going through the ISP’s internal network prior to exiting.  
    The 3 numbers you see are the individual times to get to that specific hop.  It is important to note that these numbers do not represent the difference in time between the current hop and the previous, but rather represent the cumulative time taken up until that hop.  When you look at a traceroute you are looking for the first point at which there is a high degree of variation between times (for example: 50 ms 283 ms, 29 ms) or for times that are consistently much higher than the preceding hop’s. You may also see “*” as an entry.  These indicate no reply was received from the server.  These are not necessarily signs of an issue, especially once you reach Salesforce.  Certain networks do not reply to the packets used in traceroutes for security reasons. Once you are inside the datacenter, Salesforce does the same.  If you see one hop has one timeout, that probably isn’t a problem as long as things look fine otherwise. It is important to note that your “normal” traceroute and ping results will vary based on geographic location of both your system and the datacenter.  For example, if you are in Australia and are connecting to an eastern US data center a 250-300ms roundtrip time is not out of the ordinary.  However, a 300ms roundtrip time from Australia to the Tokyo datacenter would be unusual.

     

  3. Look for odd routing decisions and node specific issues.
    ISP’s are constantly making updates to their networks such as tweaking routing decisions and adding new lines.  They do this to keep their network healthy and to try and optimize around certain traffic patterns. There are occasionally times when these changes could route your requests to Salesforce via a less than ideal path. For example,  if you are accessing a North American based Salesforce instance from California, but are being routed through Singapore and back before reaching the application, you will certainly see increased load times.  This will display in ping and traceroute results as consistently high round trip times, +300 ms, with little variance between times at each hop and next to 0 packet loss. You may also see two large jumps of more than 100ms over the span of 1 hop.  This is typical behavior of an intercontinental hop, or where your traffic crosses significant geographical distance.
    If you are based in the same region as your data center (North America for NAXX instances, Asia for APXX, Europe/Africa for EUXX) and you happen to notice a large jump in your traceroutes, you can go to [IP address].ipaddress.com to find out the geographic location of the specfic IP address where the jump is occurring.  
    If you notice this type of issue please contact your ISP as they control the route your request takes to Salesforce.
    Another example of a network routing issue is below:

     Bad trace.JPG

    In this traceroute we can see 5 *’s on lines 6 and 7.  Another thing we can see is that both lines 5 and 7 are owned by Level3 Communications and we are making a transatlantic hop.  Because it is a transatlantic hop we aren’t going to concern ourselves with the ~75ms increase in time.  What we should be concerned with is that our transatlantic hop is inconsistent.  If this were tied to a ping in step 3 you would probably see packet loss.  You would want to contact your ISP in this case and let them know what you have found.  Your ISP should be able modify the route your traffic takes to utilize a different transatlantic link or at the very least contact their partner (Level3 in this case) to pursue the issue further.

  4. Contact Salesforce Support

    If you are still unable to pinpoint any networking issues, please send all the information you have collected using the steps above to Salesforce support.  We will take a closer look at your orgs and work with our Network Operations team to assess.





promote demote