How I stopped blocking GoogleBot and passed the Mobile-Friendly Test
So I was reading a Food Blogger Pro Podcast transcript about SSL and
HTTPS, and a curiosity brewed in me to check
janepedia.com against Google Search. I typed “janepedia” into the address bar, and lo and behold
https:// prefixed it, but in red letters beneath the URL read, “Your page is not mobile-friendly.” I had to do some digging, but my
robots.txt was blocking GoogleBot…and the issue was not as straightforward a fix as I’d have expected—but it also wasn’t as complex a fix as I’d expected, so that’s awesome. I briefly feared I was going to have to dial 4367443.
Failing the Mobile-Friendly Test
Georgie codes mobile-first, so I knew that could not be an issue at all. Thinking now, I suppose I’d not feel so confident in her code’s mobile-friendliness if she didn’t say, “I code mobile-first,” so many times. So…ha, I guess it’s good to brag about yourself sometimes! That troubleshooting step was easy to tick off.
After some time, I noticed the CSS of my site wasn’t loaded. Perturbed and assumptive, I submitted feedback about this, even detailing the fact that I use responsive images because Georgie rubbed off on me, so the photos rendered by the test should have at least been 300px wide. Because I’m a visual person, here is the image code of/for the image on my about page (which I recently rewrote):
<img src="https://janepedia.com/img/pages/about/me-newt-1000.jpg" srcset="https://janepedia.com/img/pages/about/me-newt-300.jpg 300w, https://janepedia.com/img/pages/about/me-newt-500.jpg 500w, https://janepedia.com/img/pages/about/me-newt-1000.jpg 1000w" alt="Photo of myself with Newt, a miniature jack donkey; in the background are miniature donkeys in another pen" width="1000">
This is also why I find editing and uploading images more tedious than shaving my legs legally blind (because glasses + the shower don’t mix) while I’m trying to avoid scars, blisters, and whatever else might invoke an audible whine (as opposed to my typical inaudible ones paired with mere facial expressions).
If the CSS had been rendered, obviously the width of the image would be
max-width:100% regardless of whatever size was rendered, though. I mean, the CSS literally would solve all the issues. So I finally realized, Oh! Duh! and looked up—on Google Search, no less—how to even fix this. I’m not a fan of their community-based support system—there are a lot of snarky, entitled people who feel a need to parent people asking for help (or maybe I’m just different in that respect and try to fight fire with kindness and logic). It’s why I prefer smaller companies.
The partially-loaded page dilemma
From one topic, I discovered there was an option to view loading issues at the top of the scroll frame. Despite it having a yellow caution symbol, I overlooked it in favor of the big red text saying my site wasn’t mobile-friendly.
After, I deduced it was a GoogleBot +
robots.txt issue, based on the detailed table.
Resolving the GoogleBot vs.
I looked it up, ’cause last time I let bots index my files, those ended up in Google Search, and I was getting hacked left and right. I started using iThemes Security, formerly known as Better WP Security, but I stopped because I felt uncomfortable letting a plugin have that much control over my website—and after a major update, it changed every single image to not be visible to anyone. (I use Wordfence Security now, as suggested by Nancy.)
This is where it gets complicated, because I didn’t want to allow every theme (no one needs to know which themes I have on my site!) nor every plugin to be indexed. I think this will be another reason I dislike third-party plugins, because it means I have to keep up with the paths of them to allow on
Using another community forum topic response, I came up with this:
Allow: /wp-content/plugins/fd-footnotes/*.js Allow: /wp-content/plugins/jetpack/css/*.css Allow: /wp-content/plugins/recipe-card/css/*.css Allow: /wp-content/plugins/recipe-card/js/*.js Allow: /wp-content/themes/janepedia-mercury/*.min.css Allow: /wp-content/themes/janepedia-mercury/style/*.min.css Allow: /wp-includes/js/*.js Allow: /wp-includes/js/jquery/*.js
To test whether this worked, I pressed the circle with an arrow at the far-right end of the URL field on the test to refresh.
Remaining loading issues
- I removed the social follow plugin and just placed my social media into my footer.
- The remaining issue thereafter is created by Google Analytics (don’t even get me started on the support topic for that.)
I hope you find this useful if you find yourself having similar issues/dilemmas. Afterward, I pressed “Submit to Google” so I wouldn’t have to wait for unreliable bot timing. If you want to further optimize your
robots.txt for search engine optimization (SEO), WPBeginner has a detailed post about that.