Introduction to Technical SEO
Master the technical foundations of search engine optimization. This comprehensive guide covers essential technical SEO practices to improve your website's visibility, crawlability, and search engine rankings.
1. Preferred Domain Configuration
Why Set a Preferred Domain?
When setting up your website or blog, you need to specify your preferred domain. By doing so you instruct search engines as to which variation of the domain you want to use throughout the lifetime of your website.
Understanding WWW vs Non-WWW
A website (by default) is accessible with www and without www in front of a domain name.
For example, if your domain is example.com, your website can be accessed by both https://www.example.com and https://example.com (i.e. without the www).
While this is ok for users, it is confusing for search engines because they consider these as two different websites.
Potential Issues
This means that you may encounter indexing issues, duplicate content problems, and loss of page rank. To solve this problem, you need to set your preferred domain and inform search engines about your choice.
Technical Difference
Let's see a couple of examples:
URLs with www
https://www.example.comhttp://www.example.com
URLs with no www
https://example.comhttp://example.com
Domains with no www in front are also called naked domains. Domains with www can also act as a hostname which can sometimes be easier to manage when it comes to cookies, in cases where you have a number of subdomains assigned to the www domain.
Which Format Should You Choose?
The answer is simple. It depends on your personal preference. There is no SEO advantage from using one format or the other.
Three Important Requirements:
- Configure your website to 'listen' to only one of the variations
- Let Google and other search engines know what is your preferred choice
- Be consistent and use the chosen variation when adding internal links
How to Set Your Preferred Domain
Step 1: Check Your CMS
Login to your CMS and look for the domain-related settings. Most CMS allows you to specify your website URL. If you're on WordPress you can log in to your WordPress dashboard and then click GENERAL and then select SETTINGS from the left menu.
Step 2: Communicate Your Preference to Search Engines
Google decided to remove the preferred domain option from Search Console and now relies on canonical URLs. A canonical URL (aka canonical domain), is a piece of HTML code that is added in the HEAD section of a page and tells search engines which is the preferred URL for the particular page.
To check if your website sets the canonical domain correctly, open your homepage in a new browser window and then go to VIEW SOURCE (right-click anywhere on the page and select view source from the menu).
Search for the word 'canonical' and you should see a line like this:
<link rel="canonical" href="https://www.example.com/" />Key Learnings
- Having www or no www in front of your domain name is a matter of preference
- There is no SEO advantage to choosing one version or the other
- Choose one format and stick with it for the lifetime of your website
- Use canonical URLs to communicate your preference to Google
2. Robots.txt Optimization
What is Robots.txt?
A robots.txt is a text file that resides in the root directory of your website and gives search engines crawlers instructions as to which pages they can crawl and index during the crawling and indexing process.
When visiting a website, the first thing search engines do is to look for and check the contents of the robots.txt file. Depending on the rules specified in the file, they create a list of the URLs they can crawl and later index for the particular website.
Why Use Robots.txt?
The most common use cases of robots.txt are the following:
- To block search engines from accessing specific pages or directories
- To reduce server load on large websites by restricting crawling
- When you use URL cloaking for your affiliate links
Important Things to Know
1. Rules are directives only: Any rules you add to the robots.txt are directives only. This means it's up to search engines to obey and follow the rules.
2. May still appear in search: Even if you block a page or directory in robots, it can still appear in the search results if it has links from other pages that are already indexed.
How Does Robots.txt Work?
The robots file has a very simple structure with predefined keyword/value combinations:
Common Directives:
- User-agent: Specifies which crawlers should take into account the directives
- Disallow: Instructs a user-agent not to crawl a URL or part of a website
- Allow: Tells explicitly which pages or subfolders can be accessed (Googlebot only)
- Crawl-delay: Forces crawlers to wait a specific amount of time before crawling the next page
- Sitemap: Specifies the location of your XML Sitemap
Example Robots.txt:
User-agent: *
Allow: /
Sitemap: https://example.com/sitemap.xmlHow to Create and Test Robots.txt
Creating a robots.txt file is easy. All you need is a text editor (like brackets or notepad) and access to your website's files (via FTP or control panel).
Before getting into the process of creating a robots file, the first thing to do is to check if you already have one. The easiest way to do this is to open a new browser window and navigate to https://www.yourdomain.com/robots.txt
The best way to test and validate it is through the robots.txt tester tool in Google Search Console.
Key Learnings
- Test your robots.txt and ensure you're not blocking important pages
- Do not block CSS or JS folders - Google needs them to render pages properly
- Don't specify different rules per search engine bot - use user-agent:* for simplicity
- Use meta robots tag instead of robots.txt to exclude pages from indexing
3. URL Structure and Optimization
What is a URL?
A URL is an acronym for Uniform Resource Locator. In simple terms, a URL specifies the web address of a page. Each and every web page on the Internet has a unique URL (also called a permalink).
The URL of a single page has two parts: the first part is the domain (not configurable), and the second part is the path to find the page within the domain (this part is configurable).
What is an SEO-Friendly URL?
A friendly URL accurately describes the page using keywords that are easy to read for both search engines and users.
Example of SEO-friendly URL:
https://www.example.com/seo-friendly-blog-posts/Example of NON-SEO friendly URL:
https://www.example.com/Folder/X1356774/124Why URLs Matter for SEO
- User Experience: Easy to understand URLs improve the user experience
- SEO Ranking Factor: Although minor, well-crafted URLs are a ranking factor
- Click-Through Rates: URLs shown in search results attract more clicks
- Anchor Text: Users linking with the URL provide keyword context
How to Optimize Your URLs
1. Choose a Good Domain Name
- Short (2-3 words max)
- Catchy and easy to remember
- Preferably a .com, .net, or .org domain
- For local businesses, use country-specific domains (e.g., .co.uk)
2. Include Target Keywords
Help Google understand which keywords you want to rank for by including those keywords in the page slug. For example, instead of "10-amazing-health-benefits-of-asparagus-you-should-know-about", use "asparagus-health-benefits".
3. Remove Stop Keywords
Consider removing common words like "a", "the", "on", "and", "is", "of", "you" that don't add value.
4. Use Dashes to Separate Words
Separate words in a slug using a "-" (hyphen) and not any other character.
5. Keep It Short
Shorter slugs are easier to understand and faster to process than longer slugs.
6. Use Only Lowercase Characters
Avoid Capital characters in your slug but use only lowercase letters to prevent server confusion.
7. Use HTTPS URLs
Having a website with an SSL installed makes your URLs HTTPS, increases security, gains users' trust, and provides a small ranking boost.
8. Avoid White Spaces
White spaces in filenames translate to %20 in the URL, which is not user-friendly.
9. Don't Use Dates
Dates should not be part of the URL since they don't offer benefits to users or search engines.
10. Keep Folder Structure Simple
Keep your folder structure to 2 levels maximum. Each page should be accessible from the homepage in less than 3 clicks.
Key Learnings
- SEO friendly URLs are important for both users and search engines
- Configure URL structure at the beginning and rarely change it
- If you must change URLs, use 301 redirects to maintain SEO rankings
- Short, descriptive slugs encourage more clicks from search results
4. XML Sitemap Implementation
What is an XML Sitemap?
In simple terms, an XML sitemap is a file that lists all the important pages of your website that search engine crawlers should know about.
Even if you don't have a sitemap, search engines can still index your website, but having a sitemap makes their job easier, and it also allows you to let them know about pages or parts of your website they may not discover easily.
When is a Sitemap Really Important?
- For websites with many pages, a sitemap helps discover new or updated pages
- For websites that don't use good internal linking practices
- For new websites that don't have many incoming links
How to Create an XML Sitemap
To check if your website has a sitemap, open a new browser window and navigate to http://www.yourdomain.com/sitemap.xml
An XML sitemap is not a static file, but it needs to be updated automatically when a new page is added to your website or when changes are made to existing pages.
For WordPress websites, plugins like Yoast SEO can automatically generate and update your sitemap.
What to Include in Your Sitemap
The most important sitemap optimization guidelines are:
- Include pages that are important and have high-quality content
- Exclude pages that have duplicate content (archive, tag, author pages)
- Exclude pages with 'thin content'
- Organize your sitemap into sub-sitemaps for larger sites
- Submit a separate sitemap for videos if applicable
- Add your sitemap in robots.txt
- Include only URLs that Google can access
- Include the 'last modification time' field
How to Submit Your Sitemap to Google
There are two ways to ensure that Google will pick up your sitemap:
Method 1: Add to Robots.txt
Edit your robots.txt file and add a line at the end:
Sitemap: https://www.example.com/sitemap.xmlMethod 2: Submit in Google Search Console
- Login to Google Search Console
- Click SITEMAPS under INDEX
- Type in your sitemap filename and click SUBMIT
Key Learnings
- Having a valid XML sitemap is important for SEO
- Submit your sitemap once; Google will monitor it for changes
- Many plugins automate sitemap creation and updates
- You can also submit your RSS feed to speed up indexing of new content
5. Schema Markup and Structured Data
What is Schema Markup?
Schema markup is a way to 'describe' your content to search engines in a way they can understand it. This is done by adding pieces of code to your HTML that consists of pre-defined tags (semantic vocabulary) that search engines understand.
Why is Structured Data Important for SEO?
- Prepare for Future SEO: Feature gaining ground that may enter ranking algorithm
- Rich Snippets: Enhanced search results with more attractive, informative entries
- Local SEO: Provide detailed local business information to search engines
- Google Shopping: Required for running successful shopping campaigns
- Voice Search: Helps rank in featured snippets and voice search results
Supported Schema Types
Google currently supports the following schemas:
- Article
- Book
- Breadcrumb
- Carousel
- Corporate Contact
- Course
- Critic Review
- Dataset
- Event
- Job Posting
- Local Business
- Logo
- Product
- Recipe
- Review Snippet
- And many more...
How to Add Schema Markup
Manual Method: Google Structured Data Markup Helper
- Open the structured data helper tool
- Select the type of schema you want
- Enter a URL from your website
- Tag elements on your page
- Generate and download the code
- Add JSON-LD script to your page HEAD
Automated Method: WordPress Plugins
The Yoast SEO plugin has built-in support for several schemas including:
- Webpage
- Organization
- Person
- Article
- Image
- Video (Premium)
- Local SEO (Add-on)
How to Check for Errors
Use these methods to troubleshoot structured data:
- Structured Data Testing Tool: Validate your markup code
- URL Inspection Tool: Check pages in Google Search Console
- Enhancement Report: View detailed reports in Search Console
Key Learnings
- Structured data is important for modern SEO practices
- Check if your website already uses markup data
- Fix errors using the testing tool or hire a developer
- Use plugins for automated implementation on WordPress
7. Canonical URLs
What is a Canonical URL?
A canonical URL is added in the HEAD section of a page and tells search engines which is the preferred URL for the particular page. A canonical URL can point to itself (self-referencing) or a different URL.
<link rel="canonical" href="CANONICAL-URL"/>When to Use Canonical URLs
1. Solve Duplicate Content Issues
Typical cases of duplicate content include:
- URLs accessible with or without www
- URLs accessible with both HTTP and HTTPS
- Pages with print-friendly versions
- Pages with identical content but different URLs
2. E-commerce Websites
Product category pages that have the same content and use filters. For example, product pages showing the same items in different colors.
3. Content Syndication
When you re-publish or syndicate your content on other platforms like Medium or LinkedIn, use canonical URLs to tell Google the original source.
How to Find Canonical URLs
Method 1: View HTML Source
Open a webpage in Chrome, select View > Developer > View Source, and search for 'canonical'.
Method 2: URL Inspection Tool
Use Google Search Console's URL Inspection Tool to see what Google considers to be the canonical URL.
Canonical URL Best Practices
- Each page must have a canonical URL (self-referencing or to another URL)
- Only one canonical URL per page
- Canonical tag specified in the HEAD of the page
- The URL must be valid
- The canonical URL should not have the "noindex" attribute
- Avoid creating loops with canonical tags
- Use on pages with duplicate or similar content
- URLs should follow the same format as homepage
- For AMP pages, canonical should point to non-AMP version
Canonical URLs vs 301 Redirects
Canonical URLs: Users can still see both pages, but search engines are told not to consider one of them. Backlinks are consolidated to the target page.
301 Redirects: Users don't see the source page, only the target. Backlinks are also consolidated to the target page.
Rule of thumb: Use canonical URL when you want to keep the page on your site for navigation or user purposes. Use 301 when you don't want users to see the particular page.
Key Learnings
- Each page needs a canonical tag set (self-referencing or to another URL)
- Canonical URLs behave like 301 redirects for link consolidation
- For e-commerce, ensure dynamically generated pages have proper canonicals
- Use 301 redirects when you don't want pages visible to users
8. Hreflang for Multilingual Sites
What is Hreflang?
If your website is available in more than one language, you must implement hreflang. Hreflang will help Google serve the most appropriate page to your users (e.g., German pages for people in Germany) and protect you from possible duplicate content issues.
Implementation Methods
Method 1: Using Hreflang Tags
For each page on your website, you need to tell search engines the equivalent URL in all languages that the particular page is available.
Example for English and Spanish versions:
<link rel="alternate" href="http://www.example.com/en/page1/" hreflang="en" />
<link rel="alternate" href="http://www.example.com/es/page1/" hreflang="es" />Method 2: Using a Sitemap
Add language variations directly in your XML sitemap. For each page, you need to add the URLs in all available language versions.
Method 3: WordPress Plugin
Use a plugin like WPML. Install it, choose your language settings, and the plugin will handle the technical implementation automatically.
Best Practices
- Each language version must list itself as well as all other language versions
- Alternate URLs must be fully-qualified, including the transport method (HTTP/HTTPS)
- Alternate URLs do not need to be in the same domain
- If two pages don't point to each other, the tags will be ignored
- Consider adding a fallback page for unmatched languages using x-default value
Key Learnings
- If you have multiple language versions, tell Google about these variations
- Hreflang helps Google point users to the most appropriate version
- Google can find alternate versions without action, but explicit indication is best
- Proper implementation prevents duplicate content issues across languages
9. Pagination Best Practices
What is Pagination?
In the context of SEO, the term paging is used to describe a situation when you publish a big article in several parts (pages). For example, a 10,000-word guide published as a series of blog posts across multiple URLs.
Important Note
The SEO techniques described here do NOT apply to Google, but they still apply to other search engines like Bing. A better way to deal with pages with a lot of content is to use 'pillar pages'.
How to Implement Pagination
Define in the <head> of each page the 'next' and 'prev' tags. Example for a 4-part article:
First Page:
<link rel="next" href="http://www.example.com/myguide-part2">Second Page:
<link rel="prev" href="http://www.example.com/myguide-part1">
<link rel="next" href="http://www.example.com/myguide-part3">Third Page:
<link rel="prev" href="http://www.example.com/myguide-part2">
<link rel="next" href="http://www.example.com/myguide-part4">Considerations
Advantages: Pages load faster and users may spend more time on your website clicking through content.
Disadvantages: Most users prefer to have everything loaded on one page rather than clicking to read the next part.
Before making a decision, consider the experience of your users. Perform A/B testing before adopting this policy for all content.
Key Learnings
- Implementing pagination using 'prev' and 'next' tags is optional
- These tags are not taken into account by Google but other search engines still use them
- A better approach is to use 'pillar pages' for long-form content
- Always prioritize user experience over technical implementation
10. 404 Page Optimization
What is a 404 Page?
A 404 page is shown to users when the URL they visited does not exist on your website. Maybe the page was deleted, the URL was changed, or they mistyped the URL in their browsers.
Why Optimize Your 404 Page?
SEO is about improving the user experience, and a proper 404 page contributes to that goal. Most modern WordPress themes have optimized 404 pages by default. If not, you can easily improve your 404 page using a plugin or editing your theme templates.
What Makes an Optimized 404 Page?
An optimized 404 page should:
- Have the same structure and navigation menus as your website
- Tell visitors in a friendly language that the page is no longer available
- Give them alternatives (suggest other related pages)
- Make it easy to go back to the previous page, homepage, or other important pages
How to Test Your 404 Page
Testing is very easy: just open a new browser window and type a URL on your website that does not exist. What will be shown in the browser is your 404 page.
Bad 404 Page: Displays generic "Not found" message with no design or navigation options.
Good 404 Page: Provides user-friendly message, maintains site design, and offers navigation alternatives.
Key Learnings
- Don't spend too much time optimizing 404 pages
- Ensure custom 404 pages give users alternative ways to find content
- Maintain consistent design with the rest of your website
- Provide helpful navigation options instead of dead ends
11. Site Structure and Architecture
Why Site Structure Matters
A well-defined site structure makes the job of search engine crawlers easier, which means better indexing and more chances of achieving higher rankings.
With the introduction of artificial intelligence in the Google ranking algorithm, voice search, and mobile-first index, the structure of a website has gained more importance than ever.
Importance of Good Site Structure
- Helps search engines understand your content better
- Increases chances of getting Sitelinks in Google search
- Provides a great user experience and lowers bounce rate
- Makes navigation intuitive and efficient
Ideal Site Structures by Type
1. Blog Structure
Homepage → Categories → Blog Posts. Posts should be grouped into categories according to relevancy. For example, a food blog can have categories like Italian Recipes, Vegetarian Recipes, Chinese Recipes, etc.
2. Small Business / Corporate Website
Homepage → Departments → Services/Products → Blog Categories → Blog Posts. Provides information about the business, its products, services, and departments.
3. News Website
Homepage → Categories (by topic) → Archive (by date) → Articles. Group content by type and publication date.
4. E-commerce Website
Homepage → Product Categories → Subcategories → Product Pages. Can be more complex, but products should be accessible from homepage in less than 3 clicks.
How to Optimize Your Site Structure
1. Plan Your Structure in Advance
Use ideal structures as guides and adjust according to your needs. Your website should have a hierarchical structure with the homepage as the focal point.
2. Limit Hierarchy Depth
Don't overcomplicate your site hierarchy. Keep it simple with no more than 2 or 3 levels deep. All pages should be accessible from the homepage in less than three clicks.
3. Use Categories for Related Content
- You NEED to have categories
- Try to keep your categories the same size
- Add categories to the menu and homepage
- Use categories to create content relevancy
4. Use HTML or CSS for Navigation
Avoid using Javascript or Ajax for navigation. Use plain HTML tags combined with CSS for formatting to make crawlers' jobs easier.
5. Implement Breadcrumb Menus
Ensure breadcrumb menus display your site structure accurately, showing all levels from current page back to homepage.
6. Optimize Internal Link Structure
- Create internal links pointing to your category pages
- Identify most valuable pages and link to them from related pages
- From category pages, link to the most valuable pages of that category
- Use optimized anchor text for internal links
- Ensure ALL pages are reachable through internal links
7. Provide Users with a Sitemap
Include an HTML sitemap in your website's footer that shows the website hierarchy to users.
Key Learnings
- A well-defined site structure is crucial for modern SEO
- Structure should be hierarchical and easy to use
- Give search engines more clues about topics you want to rank for
- Users should reach any page by following links from the homepage
Start Implementing Technical SEO Today
Technical SEO is the foundation of a successful search engine optimization strategy. By implementing these best practices, you'll improve your website's crawlability, indexability, and overall search engine performance.
Remember: Technical SEO is an ongoing process. Regular audits and updates ensure your website stays optimized as search engine algorithms evolve and your content grows.