A Powerful SEO Tool
As an SEO professional, there are a number of tools you need to have in your arsenal to assess your website for any room for growth in your SEO strategy. From auditing to analytics, there’s a lot of data to comb through when it comes to optimising your website – and having access to the best, and most powerful, tools can dramatically reduce the amount of time spent trying to collate all the information you need in one place.
Enter Screaming Frog.
Screaming Frog is a well-respected tool among SEO professionals, offering a range of website auditing features to help support your overall SEO strategy. In this guide, we take you through the basics of Screaming Frog and share our top tips to help you use it as part of your arsenal to analyse, audit, and optimise your websites.
What is Screaming Frog?
Screaming Frog SEO Spider is a desktop-based website crawler that provides a detailed analysis of various SEO elements on a website and is a staple tool of SEO professionals worldwide.
By crawling your website (or those of your competitors), Screaming Frog gathers valuable data and presents it in a user-friendly format, allowing you to easily identify areas of improvement or assess your site’s current health.
Screaming Frog offers both a free version, which crawls 500 URLS, and a paid version which offers unlimited website crawls.
Without Screaming Frog, you’re faced with the often herculean task of manually cataloguing all pages on your website, and all the associated meta data. This can be extremely time-consuming – especially with larger websites or sites where the pages change regularly (for example, if you’re managing SEO for an Ecommerce website), and may lead to you missing an important change to your website that can impact your SEO, such as a dodgy redirect or broken link.
Screaming Frog condenses all this data into a simple dashboard or spreadsheet that can be exported and shared with your team.
Screaming Frog’s Standout Features
As an SEO tool, Screaming Frog offers a range of features that can be incredibly beneficial to your overall SEO strategy.
Crawling Site URLs
One of Screaming Frog’s core capabilities is its ability to crawl your website and report on every URL it discovers. If you’re using the free version, the crawl is limited to 500 URLs, while the paid version removes this restriction.
During a crawl, the tool collects detailed information about each page, including:
- Status codes (200, 301, 404, etc.)
- Page titles and meta descriptions
- Canonical tags
- Header tags
- Content type and size
This makes it much easier to identify technical issues such as broken pages, redirect chains, or pages that are unintentionally blocked from search engines. It also provides a clear overview of how search engines might interpret your site.
Visualise Your Site Structure
Understanding how your website is structured is crucial for both usability and SEO. Beyond the main table view, the tool includes visualisations such as crawl diagrams and directory tree views, allowing you to see how pages connect within your site architecture. These visual insights can highlight structural issues like pages that are buried too deep within the site or areas where internal linking could be improved.
Generate XML Sitemaps
Screaming Frog can generate XML sitemaps directly from your crawl data. Sitemaps help search engines discover and understand the structure of your website by listing the URLs you want indexed.
Analyse On-Page SEO Elements
The tool also provides detailed analysis of key on-page SEO elements. It scans your website for important optimisation factors such as:
- Title tags
- Meta descriptions
- Header tags (H1, H2, etc.)
- Image alt text
- Content length
By highlighting missing, duplicate, or overly long elements, Screaming Frog allows you to quickly identify areas where pages may need further optimisation.
Internal and External Link Analysis
The crawler scans both internal links (links between pages on your site) and external links (links pointing to other websites). This makes it easy to identify any broken links, redirect chains, orphan pages, and plan improvements to your internal linking structure.
How Screaming Frog Can Transform Your SEO Strategy
Screaming Frog can significantly enhance your SEO strategy by providing comprehensive site audits that identify and rectify various issues affecting your website’s performance. The tool’s keyword mapping and on-page optimisation features allow for detailed analysis of meta tags, headers, and on-page content, ensuring that keywords are mapped effectively to improve search rankings. Additionally, Screaming Frog’s ability to analyse your link profile helps you evaluate both internal and external links, identify broken links, and assess anchor text distribution. This thorough examination is crucial for maintaining a healthy link structure.
Screaming Frog aids in enhancing your site’s structure by evaluating the hierarchy of your content and identifying opportunities to improve navigation, thereby enhancing user experience. The tool’s seamless integration with other essential platforms like Google Analytics and Search Console further enriches its utility, providing a holistic approach to managing and improving your website’s SEO.
How to Use Screaming Frog
To use Screaming Frog effectively for your SEO strategy, start by opening the application and initiating a crawl of your website. Enter your website URL in the search bar and click the ‘Start’ button to begin the crawl. As Screaming Frog scans your site, it will collect a wealth of data across various dimensions.
Once the crawl is complete, explore the different tabs to gain valuable insights. The ‘Internal’ tab displays all the internal links on your site, while the ‘External’ tab lists links to external websites. The ‘Response Codes’ tab provides information about the HTTP status codes returned by each URL, helping you identify pages that are returning errors. The ‘Directives’ tab shows the meta tags and directives such as noindex, nofollow, and canonical tags, which are crucial for SEO. The ‘Custom’ tab allows you to configure and view custom extractions based on your specific needs.
As you navigate these tabs, pay close attention to key SEO concerns. Look for broken links that need fixing, unoptimised pages that require better keyword targeting, and duplicate content that could harm your SEO efforts. Addressing these issues will help improve your site’s overall performance and search engine ranking. By systematically analysing and acting on the insights provided by Screaming Frog, you can enhance your SEO strategy and ensure your website is fully optimised.
Screaming Frog is a robust tool with plenty to offer, and the best way to find out what you can do with it is to practice. We’ve tried to provide a quick, easy overview, but we’re only just scratching the surface. For more information, the Screaming Frog website has some great guides.
Get Support with Your SEO Audits
At Bespoke Digital, our expert team of marketers use various tools, such as Screaming Frog, in order to provide the very best SEO services for our clients. For more information, why not reach out to us with any questions?
