This website uses cookies to make your experience better. You can read more about cookie terms here.

Unable to hide old product pages indexed on Shopify robots.txt?

"An easy way to do it, no apps involved."

Shopify does not offer editing of robots txt and maintains the files itself. This is troublesome for SEO but you can hide the old product pages by just adding the ‘noindex’ tag into the required page.

Sadly, as of 2021, we do not see this issue of editing robots.txt being resolved ever since it was raised. However, in order to serve Google better, Shopify store owners have found ways to address this issue.

In this blog post, we will share what is robots.txt and how to hide old product pages on Shopify robots.txt file and also how to verify if the robots.txt is updated.

Table of Contents

What is robots.txt and how it could affect my store?

Google uses robots or we can say ‘crawlers’ to find your store product pages. When your pages are crawled by the Google robots, it helps in ranking on Google. Simply put, if Google can’t find your pages, it won’t be able to show it search results.

Every store has its robots.txt file which tells Google crawlers which pages should be crawled and which shouldn’t. However, this doesn’t mean robots.txt file keeps your pages ‘not to show’ but it avoids overloading of your website/store.

Probably, your store has too many pages indexed even if you don’t think you do, must check. You will get surprised. Point is, if a search engine crawls any store, it would crawl every single page.

If your store has a lot of pages, the search engine bot/crawler will consume more time which can negatively affect your store rankings overall.

Do you know Googlebot has a crawl budget?

Crawl budget is the number of URLs a bot crawls. This budget depends on two factors. First the crawl rate limit, here’s how Google explains it:

Shopify robots.txt

Second is crawl demand and here’s what it means:

crawl demand and crawl budget

In our use case where we don’t want old or outdated pages to show in Google results, we can use the ‘noindex’ tag in any of the required product pages.

What is a noindex tag?

A tag which tells Google not to crawler any specific product page and usually embedded in code like this:

You may be thinking where to add this code when Shopify doesn’t allow it. Hold your horses, there is a hope. First, you need to locate your robots.txt file in the Shopify store.

Where can I find robots.txt file?

Find your store’s robots.txt file in the root directory of your store’s primary domain name.


List of pages must be blocked by robot.txt

Why is Google banning indexing my product page URLs?

The obvious reason is to prevent duplicate pages from indexing.

Oftentimes you misread the robots.txt file. There could be several instances when Shopify won’t allow inappropriate URLs.

There could be many product pages which are duplicate – just think that you have 10 tags. There are 1,025 combinations of those 10 tags. And most of those combinations are nonsensical or don’t add a lot of value — that’s why Shopify is preventing them from being indexed.

‘Noindex technique is recommended by Google’ for hiding/removing old pages

One of the best ways to hide your product pages from being indexed is using a noindex tag as we were referring to before.

1-Go to your admin dashboard and enter into themes section. Select ‘edit code’.

noindex technique

2-Choose the theme you want to edit which is theme.liquid.


3-Enter the following code in the head section. We used /baby-clothes’ in the category name as shown in the screenshot.

{% if product.url == ‘/products/categoryname’ %}

<meta name=”robots” content=”noindex”>

{% endif %}

content editing in theme.liquid

Hit save on the top right corner of the screen.

Verify if the robots.txt is updated by using the tester tool

Finally test your robots.txt file in order to be sure that old or outdated pages are not ranking anymore. Google provides a free tool i.e. robots.txt tester as part of the Webmaster tools.

Add the updated robots.txt there and test. Remember, changes in actual code may take a day or so.

Final verdict

SEO of a Shopify store is often tough given the fact that you can’t edit the robots.txt file directly. You have to look into hacks in order to do so. However, editing Shopify robots.txt may be a daunting task if you have to hide multiple product pages. Just make a list of all the stale/duplicate product pages and convert the page names into page handles. Adding {% else if handle contains ‘new-page-handle’ %} in the head section may not be daunting if there are only a few product pages you wish to hide.

In case you are overwhelmed with the duplicate product pages and can’t handle the robots.txt file yourself, we are always happy to help.

Subscribe to our Newsletter.

Thank you! Your subscription has been added to our newsletter!
Oops! Something went wrong while submitting the form.

Related Blogs.

How to apply Shopify multiple discount codes at checkout?How to apply Shopify multiple discount codes at checkout?
People love discounts, so your customers would love them too. Don't know how to apply Shopify multiple discount code options? Use this guide for help.
How to add video to product images Shopify?How to add video to product images Shopify?
Videos are used to engage customers and demonstrate a product. Can't add a video to product images in Shopify? We have some methods to fix this issue.
How much does a Shopify expert cost? US vs Non US developersHow much does a Shopify expert cost? US vs Non US developers
"How much does Shopify expert cost?" We have a guide on Shopify development costs involving various regions like the US, UK, Australia, etc.


You have an idea? We handle all the rest

Drop us a line.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.


500 Grant Street, Downtown, Suite 2900,
Pittsburgh, PA 15219, USA

+1 (412) 475 9053


2 West Canal Bank Road, Lahore 54000,

+92 (322) 452 4442

United Kingdom

Suite A12, 56 Wood Lane, London,
W12 7SB, UK

+1 (93) 606  7875

Project inquiries

It’s not about ideas
It’s about making ideas happen
Copyright 2022 - Phaedra Solutions.
All Rights Reserved