Roast My Site #3: Website Migration Edition
Bartosz: Welcome, everyone. We have Aleyda today with us. Aleyda is the person when it comes to migrations, obviously, along with me and Arsen. We invited Aleyda because she's so well-known for her talks on migrations: website migrations, CMS migrations, and all that.
Arsen: Today we'll be talking about migrations. There's all kinds of different migrations. Aleyda, what kind of migrations do you see the most?
Aleyda: I think that it's important to specify as much as possible because we tend to get clients who say, "Oh, I need help with a migration," but a migration can be something very straightforward. For example, if it is an HTTP to HTTPS type of migration, it tends to be very straightforward because the URL structure doesn't tend to change much, it is a protocol change.
But then we have the type of migration when we do re-brandings when we merge websites because maybe we were acquired or maybe because we are trying to consolidate the international web properties.
I think that the higher the complexity tends to happen when there are many mixes of this happening at the same time and the bigger the structural change it is. For example, you do have a very well-established domain that had been for, I don't know, 15 years and you have all of the authority in the world but you need to and you are pushed to migrate to another domain that is completely new.
Bartosz: In my opinion, migrations are very safe if done well. What are your thoughts on that? Because I hear a lot of SEOs, a lot of maybe not SEOs, a lot of website owners are scared of migrating to a different CMS, different domain.
Aleyda: It shouldn't be like that as long as you have a real purpose that actually requires a migration. That is what is super important and fundamental to get an SEO involved since you are thinking to migrate, to verify there's a real case for a migration, if it's really needed.
I think this is one of those cases where we are going to see that the actual negative impact might not necessarily be the migration, but the update at the end of the day. This is the tricky thing that in most cases, we think, "Okay, we isolate those are factors. I know what I am changing here." But then after an update happens or something else happens or maybe your competitor, all of sudden gets all of the leads in the world.
Arsen: This is a roast. We will be making fun of your website politely. We'll try our best to be as kind as possible. We are working with third-party data when we're looking at things. We're using tools like SEMrush that don't have access to your analytics or search console data. We're going off of what we're seeing with these third-party tools. Take everything we tell you as recommendations.
Aleyda: But you know what? We don't know the context, that is important. We know that the cases that we saw are 100% avoidable if the SEO had power.
Aleyda: The actual power of the SEO was the decision-maker. Unfortunately, I have seen cases that the client decides to do whatever they decide to do despite the recommendation of the SEO.
Aleyda: Let's get started.
Bartosz: Aleyda, do you want to begin?
Website #1: Redirecting Mistakes and the Importance of One-toOne Mapping
Aleyda: Yes. I will show my screen now. The first website that we are going to be roasting and I am going to zoom a lot. This website is a Spanish-speaking website targeted to Argentina, which actually Tienda Naranja.
They moved domain that used to be directly in a .com like this and its own domain, but they moved to a sub-domain on their naranja.com as a store. Tienda is a store in Spanish, by the way. This is the thing, when I started seeing this case, it was a little bit evident for me that both websites were rather new. They were launched last year based on SEMrush data.
It seems that they started to get traction and real activity, traffic activity since last year and they grew at the same time. We can see already the impact and realistically, we can see how here the sub-domain always had much more traffic than the other one.
I guess (assuming a lot because we don't know any insider information) that they decided that instead of having the two and coexisting and at the end of the day are only attracting the most of the traffic to the other website, they were going to consolidate them and migrate them. This is what happened: tiendanaranja.com, we can see how it started to decrease at the end of last year.
The migration happened over here and at some point, at their peak point here in July 2018, it used to have 174 keywords in the top three, more than 2000 between position four and 10 versus what is happening right now. Even if they already migrated in November, it seems that they haven't been able to recover at all from the migration.
The first step that I always do when I don't have all the data is to go back in time. I went to SEMrush and I selected the date from July 2018, for the old property, for Tienda Naranja, to understand pages they were ranking for which keywords to understand if it is because the redirects were not implemented correctly.
I really wanted to go through all of the URLs that used to get any traffic. What I did was to export this list of URLs and I did a list crawl with Screaming Frog. You can do that with any SEO crawler. All of these URLs were the URLs that used to get rankings and clicks based on SEMrush data.
Then I saw this for all of them; take a look. All of these URLs, tiendanaranja.com, yellowstore.com that are 301 redirected, all of them to the homepage. To the homepage of the new location, not to that new URL versions in the new locations.
All of them are like this. This former URL used to be ranking for a phone-related keyword. It's 301 redirected which is good, but it's 301 redirecting not to its new version and in the new location. The whole URL structure is replicated here and I don't know why, maybe it was a technical restriction. All of the URLs, as you can see here are redirecting to the homepage and this is a big no-no.
Another example here of a former Samsung smartphone, it's very descriptive even for English speakers. A URL that should be mapped and redirected to the new smartphone Samsung page in the new location, in a new subdirectory, it is redirecting to the homepage.
Unfortunately, there's no way, even if you redirect, that you will be able to recover the former rankings that you used to have here at the peak because this website was ranking for all of these different types of keywords in a very granular way with all of these internal pages. Now they are already redirecting to the homepage, the homepage will never be as relevant for them as their previous URLs.
Whatever type of product they provide, they will never rank with the homepage. One-to-one mappings here are critical. What you do need, and I actually have a very basic graph here for the minimum viable web migration to comply with the fundamentals is that for each page that used to bring any value in the past, traffic, external links that were indexable, that were crawlable, it's important to put them together and map them with a new URL version in the new web structure.
Otherwise, even if we redirect, we cannot expect to keep the previous relevance and rankings for the old queries that the old page used to rank. This is critical. The way that I can see that you can actually recover from this is fixing the redirects to do one-to-one mappings because you do have the URLs in the current structure. It's not that this Samsung pages or electrical product pages don't exist anymore in the new store. They do exist but they are not correctly mapped with the old ones.
Bartosz: I was going to try and do the screen sharing and then go next.
Arsen: I noticed the same thing. I extracted the URLs. I ran them through Screaming Frog and I did notice a lot of mapping issues. Even though there is content, they're redirecting the majority of the URLs to the root. Even though when we navigate through the website, through the new site, we can see that there are opportunities for one-to-one mapping for certain products, even categories that have historically ranked very well and had traffic. Absolutely, one-to-one keyword mapping before the redirect is so important during migrations.
Bartosz: You can put anything you like after tiendanaranja.com and it's going to redirect to the homepage. Any URL that you have in your wildest dreams, it's going to redirect. Also, what's actually quite funny, the robots.txt also is redirecting from the old domain to the new domain, which if you guys know is probably not the best idea because then it's not treated as robots.txt.
Bartosz: As Aleyda said, it's basically because of migrating everything to one domain. That's everything from me.
Arsen: When you do a site search in Google for the old domain, you still get about 21,000 results. And yes, they only migrated in November. But then when you take a quick chunk of this and drop it into Screaming Frog, they're all 301-ing, but they're 301-ing to the homepage, to the root of the new domain which is very confusing for Google.
Another thing that I observed on the site, and I'm a stickler for this, organizational issues. This is exactly the same page, but based on how I navigate to that page, the breadcrumbs are different. I would prefer these breadcrumbs to be static since you are putting all of your URLs, product URLs in the root, these breadcrumbs will help Google and users to really understand how your website is organized and it also creates logical, informational, navigational paths.
This site does need a little bit of a reworking post-migration audit, redirect mapping, and then definitely organizational information architecture reworking on this website as well. And that's all I have on this one.
Website #2: s Loss of Relevance from Google Updates & Too Many Redirect Hops
Now let me stop sharing here. All right. Who do we have next? Who's next on our list, Aleyda?
Aleyda: It's TravelFreak. Yes?
Aleyda: It's a little bit of a different case, much more complex, I will say, than the previous one that was very fundamental of badly implemented migration. This case was a website migrating from the .net to the .com, I guess, and I can only imagine that it was because the .com become available at some point or was sold to the person and say, "Of course, I want the .com instead of being on a .net.
We can see here already that this domain, the .net, already had to suffer, take a look, from the July, August 2018, core Google update. At some point, they started afterwards to recover little by little, but we can see here how they have been already hit by one of the very well-known Google core algorithm updates that happened in the summer of 2018.
And then they migrated, if we can see here, a year after, August, September. During this point in time in September, they recovered. It's interesting that this was sent as an issue of migration because I am going to show you how in general the migration is really not that bad, and we can see already how the website here started to lose the rankings with the previous domain and started to regain it already on a steady basis.
But we can see how between October and November, in November, it started to lose again a lot. Take a look, an important loss. And I guess that the person who sent this thought, "Okay, my migration was bad. This is why I cannot recover."
Let's go back in time. Let's understand how the website in the .net before was attracting rankings and traffic. We can see that at the peak time here in July 2019, this website was ranking for more than 700 keywords in the top three. 2800 between position four and 10 already. A couple of days ago I checked, it is only 132 keywords for the .com here ranking in the top three. Take a look at the difference; it's huge. And then for those keywords, we think of position four and 10, only 800 keywords. They have unfortunately lost a lot here.
I wanted to understand for which terms the old domain was ranking and all of the URLs that used to get, so you have them here, all of the URLs that used to get rankings if they were correctly redirected. The final status of the page in most cases is indexable. The page where the old ones are being redirected only in a few cases is non-indexable because they have some client errors going on, but it's definitely not massive.
In most of the cases, take a look, there was an old URL that was called travelfreak.net/places-to-visit-in-Argentina and it's going to exactly this exact same URL in travelfreak.com. These pages feature rather the same content and it has been taken to the new web structure.
Take a look: this takes a lot of my attention. They were attracting a lot of the highest share of the traffic before we're being attracted by this particular set of pages that were not necessarily the most connected one from a topical perspective to the website.
It was very simple for me to see that those all very popular pages that you structure rankings and traffic to these pages have lost a lot of visibility and ranking. I went to check these top pages in SEMrush and I could see that the decrease had actually already started before the migration.
The migration very likely only made it a little bit worse, but yeah, we can see already a negative trend going on with this particular page. It was ranking in position one. This particular page was ranking for outdoor clothing brands in position one and this is a very popular term. We can see how nowadays it went from position one and now it is in position eight.
It's not bad at all, but of course, it has lost a lot of clicks and we can see that those pages are outranking it. Many of them are stores, right? They are very transactional-oriented and the one that is informational, this Business Insider very highly authoritative.
I really do believe that this is one of those cases that we have seen happening during Google updates that is about intent and is about mixing the variety of the pages that are being ranked in the top position in order to better connect with the biggest players that could be transactional, can be informational. Unfortunately, TravelFreak here doesn't have the same type of authority as many of these other players.
It might be seen as purely informational. More prominence has been given at this point to more transactional type of websites, as we can see here. They're connected with this particular topic too. TravelFreak is not necessarily a highly authoritative website for clothing.
Something similar with Airbnb coupon codes, that was one of the other top pages. Many of the top pages that this website used to have, it used to ranking in position one and then in the top three positions, top-five positions for coupons or something like that.
When you go back in time and the date, and we can see in July, it had already dropped to position 11. This has nothing to do, I'm afraid, with the migration. This website was already losing a lot of their top terms and rankings with previous core updates that happened in the past.
What this website needs to do is to not necessarily rely on topics that are not so connected or so close from a travel perspective that is their domain, their aspect of authority. Go for those topics that they can actually grow in relevance and authority.
And that they can see that Google has really given prominence and visibility to these types of websites because it's not only about content but the format, the intent that that is going to satisfy. And yes, it's more of doing an audit from a structure perspective, topical relevance, etc. It's more about the quality and relevance and authority of the website rather than the migration
Bartosz: There are definitely some issues with how they migrated.I'm not disagreeing with you, Aleyda, in any way, I'm guessing it might actually be a mix of some of those. I just focused on some of the interesting issues here that are definitely something to look into.
I see a very sharp drop and looking at this chart, that's why I didn't assume this is an update thing because if you look at that, the date correlates very, very closely to what happened. It's basically one-to-one, the same day around September. It dropped sharply not to recover fully.
A lot of basic problems like all the HTTP URLs, they redirect to HTTPS on the old domain, then they go to a new domain. But this gets worse because there is some kind of a structure change on top of all that and it was done partially, I'm guessing, at the same time. That's just my assumption. I don't know.
These images here, the old images are not going into the right place. Those images will be very, very difficult to track back to the old domain because they are not in Google, I'm guessing. You can see that some of the old URLs are going to the homepage. Some of those are extremely, extremely complex.
If we look at one URL, one-to-one, so pictures of Greece migrated to pictures of Greece, that's the landing page that had quite a lot of traffic. It didn't recover. I would assume that all of these URLs that weren't migrated properly are somehow dragging down the old URL because this happened in September. That might be a secondary drop.
I've created a very simple to-do for them. This website 100% needs to scrape Google search results or get some old sitemaps. Every single URL you can get from the old structure going through your server logs and monitor every single redirect, every single 404, every single weird URL that goes here and make sure that you don't go through five loops of redirects.
One thing to do is HTTP to HTTPS on the old domain. It should go straight from HTTP to the new domain. All the sitemaps, redirect chains, 404s, non-301 codes, which they have quite a bit. I would go after the methodology they used to map all the URLs because they change something within the structure and if we go back and figure out how they did it, I would be sure that they can go straight from .net/URL to .com/URL without protocol change and all that.
Arsen: Okay. What would be the steps, Aleyda, in your recommendation? What will be the steps for someone who's trying to figure out whether this fluctuation or this loss happened due to migration or due to other things that are happening, like environmental things or like an algorithmic change?
Aleyda: I will say, urgently, I will say fix the ones of the migration because those should be, I think, lower hanging fruit, right? Changing the rules of the redirects to avoid so many hops, changing a little bit the unnecessary URL structure changes, for example. If it is not really necessary to change...why change? If you can keep that as it was working well before, why change it?
Making it very straightforward, one-to-one mapping, but with the little number of hops as possible, that is super important. Then on the other hand, from a topical perspective, what I am afraid here is that this website was profiting for a long time, ranking for topics that we could see, Airbnb coupon or Uber.
Arsen: Handling of internal redirects during a migration, we all know that you gotta keep it under five, and the reason for that is that Google will not follow past that fifth hop. But then there's also been talk that it only doesn't follow past the fifth hop on the first visit, that it will follow past that fifth hop on the second visit when it returns.
Bartosz: I just want to say how ridiculous this conversation is with fifth hop because we should have one hop.
Aleyda: I agree, 100%. That penalizes speed...penalizes the user experience. How much value do you lose in the meantime with every hop? So I will say it's like it's not, "Oh, Google has said that they can hop up to five. It's okay." It shouldn't be like that.
Bartosz: Especially with the modern technology that we have for the last 10 years, like crawlers, we can easily see which URL redirects where, even with like million hops and just change it to a redirect within a few clicks.
Aleyda: It's important to have internal data that is the best and then to double-check with a couple of external trackers too to correlate data, to validate data. Then another very important thing is that if this person working on the website had seen that they had been already dropping rankings and traffic as I could have seen already happening in 2018, instead of getting into a migration that is a lot of work, I would have put that effort instead on trying to improve my content and my authority, my topical relevance, etc.
Website #3: Traffic Loss from Rebranding and Typos Causing Redirect Errors
Arsen: Let's move to the next one. Aleyda, who's next?
Aleyda: The next one is a good one. Thank you.
Bartosz: My favorite one too. Yeah, sorry. Go ahead.
Aleyda: These are rebranding from letslivehere.co.uk to bricksandmortargroup.co.uk. I'm going to switch to there, to the UK index here. Yes, we can see here how it started to decrease here. It happened around April, May specifically and then there is again April, May started to increase in the other one. This is the thing, you don't see much of a difference here and that that is why it's important to have like an overlay between the two because you can actually see the difference.
I could see that there were definitely issues with the migration. One important aspect when there is a rebranding is also an additional layer, I have to say, of complexity here. We can see here how the old website was attracting a lot of brand searches.
It's important that, for example, in this particular case that the new domain will have enabled a landing page mentioning the rebranding. For example, a landing page internally saying or mentioning the name, Letslivehere or previously Letslivehere. The user can continue their browsing experience in the new website, but it's important to keep getting this old traffic, these old rankings.
They haven't, in any case, recovered the old rankings, the old traffic. I did, again, a little bit of what I was doing using SEMrush because I thought that this was a SEMrush show. I focused on SEMrush tools here mostly. I went back in time, March 2019, to see how they were ranking before; which keywords, which URLs and I get the basics of validation.
I went and checked if the old URLs...were effectively 301 redirected. 29% of the old URLs are redirecting, but there's also, let's see, 16% of the old URLs that are not redirecting that still exist. They are returning 200 OK HTTP status.
Then there is a nontrivial number of 53% here in red showing 404s. Many of these old property pages are showing errors, not redirecting at all, which is not good. Not only from a user perspective but also because the old URLs value is not passed.
I crawled it and we can see here some very important URLs that used to attract a lot of nontrivial rankings were showing errors nowadays or were 200 ok. Take a look, this was a very important old page in the old website and they were ranking in position first for student housing Newcastle that had 1,000 searches per month and this other one 2,900 searches per month, student accommodation Newcastle. This page was very important.
I wanted to check what had happened with that particular page that after the home page, it seems to be very important. It is 301 redirected to “find a property”. With this particular internal search, “find a property”. We can see here, this is the URL, find a property, it’s not relevant at all for the old keyword. It's appalling to me to see how potentially the second most important page that was, take a look, ranking for all of these terms and was bringing all of this value to this website was redirected like this.
I will definitely go and fix the 404s, the old URLs that are not redirecting and if there is also a single page that I will prioritize to redirect properly to a new relevant version is this one. Again, it's not only to redirect and that's it, but redirect to a relevant new version that keeps the relevance and allows the website to keep all rankings it used to attract. That's it for me.
Arsen: I observed the same issues. Pretty much reinforcing what you were saying, Aleyda. A really poor job was done with mapping this.
If we take a look at the old domain. We're looking at the UK database. I went back to March just to take a look at some keywords before all these fluctuations happened. What we're seeing here is, I grabbed...all the top 10 ranking keywords, extracted their URLs and did a crawl. I'm sorting by student because just like Aleyda found, I found the same thing. Student accommodations and student housing, Newcastle; these two key phrases that have historically ranked very nicely for them on the old domain.
Mapping was not handled here at all, a lot of topical focus confusion and I think the way they're structuring things with canonical also needs to be reworked. That's what I have on this. Bart?
Bartosz: Obviously, this migrated around March 2018 and they never recovered fully. Looking at some of those, they have a nice bunch of 404s, which was already mentioned. There is 11% of 404s. I exported all of the 404s. You can download those without any weird steps by following this link to Google Sheets, but there is something very meaty about this website that I found.
Let's go straight to the meat. Looking at those 200 codes that we see, actually 73% of those URLs that are properly redirected, they go to soft 404s. Aleyda, I am going to let it sit for you because it is getting way more interesting. So 75% of URLs go to soft 404s and this is how it looks like. It's not about the structure because you can see there are different parts of the website like blog, like find-a-property.
The whole problem was caused by a typo in all of those URLs. If you look at this URL and proper URL, they missed a trailing slash, like a slash.
Aleyda: Oh my God.
Bartosz: If you look at that, there are hundreds of those. Just let me repeat that there are a lot of typos. It's not only a slash issue because there are some other typos.
They slashed the rankings just because of that. You just fix those slashes, make sure that you don't have those soft 404s because if you would have 404s, you would know that you broke, to say it nicely, this migration because you don't even see 404s when you crawl that because they are soft 404s. As you can see here, your 404 page without the slash is a 200 OK code page.
That's the problem behind everything and I'm so proud to find it. To-do for them: full scrape of Google search. This is, again, just like the previous domain because you need to figure out which slashes you forgot about. Get rid of all of the soft 404s, crawl all of old sitemaps and all of the proper work with the migration.
Key Takeaways on Successful Website Migrations
Aleyda: Just to reinforce a little bit, I think that what is important to take from this is that you need to have the goal of your migration very clear to understand that it is something that is actually going to be profitable for you, that is fundamental.
If you are already suffering after every core update, maybe there are other types of issues that you need to fix and prioritize before getting into another complexity that will only make things worse. Remember if you're doing a rebrand to keep the old name also somewhere in your new website to keep the rankings for the old one coming for a while. Once people realize that you have changed the name, it won't happen anymore but at least for a while do it well.
And then 301 redirects with the fewest amount of hops. It should be a one-to-one mapping. Crawl and identify as much URLs that you have. Crawl all of new URL structure, the one that you did have many years ago.
Validate even if you supposedly did a good mapping, validate because of the issue, for example, the Bartosz showed; a typo. A slash because the programmer was, I don't know, maybe not concentrating well enough or something.It's important that you always validate before launching and after launching. Don't assume that just because you establish the mapping, it is going to be implemented well. I think that if you do this and you monitor well closely and verify again before and after, you should be more than okay.
Bartosz: Thank you so much.
Arsen: I think this is wrapping us up here. Thank you so much for joining us.
Bartosz: Thank you, Aleyda.
Arsen: Thank you for bringing all of this knowledge and power to our roast.
Aleyda: Thank you for the opportunity, it was fun.
Bartosz: I feel like we as SEOs need to educate people that migrations are safe, even complex ones, even multimillion pages are safe to migrate. But you need to have tools for that. You cannot just do that and map URLs based on an Excel sheet. You need to have a lot of data beforehand because once you press the button, that data is gone. Just reach out to Aleyda, reach out to Arsen, we're going to help you out with that.
Aleyda: Definitely. And you need to do it for the right reasons.
Arsen: Thanks, everyone.