How to Find Broken Links on a Webpage with Selenium Webdriver

As an Automation Tester you get to play with links present on the website. Links are one of the most important part of the webpage.

1. What is a Broken Link?

A broken link is a URL which is not working or not reachable. There are multiple reasons for it's dis-functioning. There are different http error status codes which the browser shows when the link is broken. These error codes have different meanings. Let's take a look at the different http status codes.

• 200 - It means success, link is working.
• 404 - It's the most common one, which mean Page not found.
• 403 - Authorization is required to access the page. 
• 400 - It has different meanings, bad request, bad host, timeout etc.
• 500 - Internal server error.

2. Why a link broke?

A link might not be working due to many reasons, like.,

• Server is down which is hosting the URL.
• Might be a human error where wrong URL is inserted in the html code by mistake.

3. How to write Selenium Code to find Broken links

Selenium Webdriver has the ability to find all the links present on the web page and also to check whether they are working or not. Otherwise, it could be a tedious task to check manually that all the links present in a website are working or not. Let's take a look at the logic which we use in our Selenium code:

1. Get all the links present on the web page on the basis of html anchor page <a>, which is used for creating a link on the webpage.
2. Store all the links inside a list.
3. Send http request to each link and verify the response received.
4. If the response code is 200 then the link is working and if response is other than 200 then link is not working.

Let's divide our solution into two parts:

1. Write Selenium Code to get all links on a web page
2. Write code to verify that those links are working or not

3.1. Selenium code to Get All Links from a Web page

public class GetAllLinks {
 
 public static void main(String[] args) {

 // Initialize Webdriver Object (Update your system's path)
 System.setProperty("webdriver.chrome.driver", "D:\\mydir\\chromedriver.exe");
 WebDriver driver = new ChromeDriver();

 driver.get("https://phptravels.com/demo/");

 // Store all link elements (anchor tag elements in html) in a list
 java.util.List links = driver.findElements(By.tagName("a"));

 // Print no. of links stored in list
 System.out.println(links.size());

 for (int i = 1; i<=links.size(); i=i+1)
  {

  // Print text of all the links
  System.out.println(links.get(i).getText());

  }

 }
 
}

Code Explanation:

1. Open the webpage.

2. Create a list of type WebElements' and store all elements with tagname 'a' in it using 'findElements()'

3. Iterate over all the links using list size as it's maximum value.

4. Get the text of the link by using getText() and print it.

Now, you have all the links in the list, you can perform different operations on it and put different checks on it. 

3.2. Write code to Finding broken links on a webpage

Now we already have the collection of all the links, next is to check whether those links are broken or not.

Links are fetched in Request-Response manner i.e. when we request a link from our browser if link is not broken we receive Response Code: 200 from server. So when browser receive 200 it means link is up and running & browser fetches data of that link from server to browser.

Now how to verify Response Code is 200. Use RestAssured Library to test if Response code is 200.

Library rest-assured/rest-assured

Add following code to the above program of get all links:

URL url = new URL(linkUrl);

//Now we will be creating url connection and getting the response code

HttpURLConnection httpURLConnect=(HttpURLConnection)url.openConnection();

httpURLConnect.setConnectTimeout(5000);

httpURLConnect.connect();

if(httpURLConnect.getResponseCode()>=400)

{

System.out.println(linkUrl+" - "+httpURLConnect.getResponseMessage()+"is a broken link");

}    

//Fetching and Printing the response code obtained

else{

System.out.println(linkUrl+" - "+httpURLConnect.getResponseMessage());

}

Handle Multiple Tabs in Selenium << Previous     ||     Next >>  Upload/Download File in Selenium