Reporting and diffing indexed URLs across runs
· 3 min read
Track which URLs Google has indexed over time with gsc-indexer's -report and -diff flags, and watch coverage shift between runs.
Want to know whether your pages are actually getting indexed as time goes on?
Use -report to snapshot a run and -diff to compare against a previous one.
Snapshot a run with -report
-report <dir> writes four files into the directory you name:
./gsc-indexer -creds sa.json -batch urls.txt -report ./report
indexed.txt: URLs Google has indexed.not-indexed.txt: everything else (not indexed, or errored).summary.json: counts + the full URL lists, in machine-readable form. This is the baseline-diffcompares against.results.json: the full per-URL detail for the run — inspected-at timestamp, verdict, coverage state, Google's chosen canonical, the sitemaps the URL was found in, and the referring URLs Google knows about. You get this whether or not you used-json, so a normal terminal run still keeps its raw data on disk.
The run also prints a summary to your terminal:
SUMMARY: 87 indexed, 13 not indexed of 100 URLs
→ ./report/indexed.txt
→ ./report/not-indexed.txt
→ ./report/summary.json
→ ./report/results.json
Compare runs with -diff
Later, re-run and point -diff at the saved summary.json:
./gsc-indexer -creds sa.json -batch urls.txt -diff ./report/summary.json
This prints what changed since the baseline:
▲ newly indexed: https://www.toolsura.com/post-new/
▼ dropped (was indexed, now not): https://www.toolsura.com/post-stale/
...
DIFF vs ./report/summary.json
indexed: 87 → 90 (+3)
not indexed: 13 → 10 (-3)
newly indexed: 4 | dropped: 1 | unchanged: 95
▲ newly indexed: URLs that moved from not-indexed to indexed.▼ dropped: URLs that were indexed before but aren't now.
Update and compare in one pass
You can write a fresh report and diff against the previous one at the same time:
./gsc-indexer -creds sa.json -batch urls.txt -report ./report -diff ./report/summary.json
The diff uses the previous summary.json (read before it's overwritten), so
this is safe to run on a schedule.
Next
- Large runs timing out? Throttling and quiet mode.
- Broken run? Troubleshooting.
Sources
- Google Search Console
- URL Inspection API documentation
- Google Search APIs overview
- Service account OAuth (Google Identity)
- About service accounts (Google Cloud IAM)
- Verify site ownership
- Property types: URL-prefix vs Domain
- URL Inspection tool help (request indexing)
- Indexing API (JobPosting/BroadcastEvent only)
Written by
Abhay Khant
Hi, I'm Abhay Thakor
I'm a developer, builder, and technology enthusiast interested in turning ideas into useful products.
I enjoy working across web development, cybersecurity, automation, SEO, and content creation. Rather than only learning technologies individually, I like putting them together to build things that people can use.
One of my main projects is Toolsura, a free online utility platform that brings practical tools into one place. I'm building Toolsura as a product and a learning project where I can explore development, infrastructure, SEO, content distribution, automation, and product growth.
What I Build
My work currently revolves around several areas:
Web development with Next.js, React, Tailwind CSS, and modern web technologies
Backend development with Go, APIs, databases, and server-side systems
Automation using n8n, APIs, Docker, and connected services
SEO and content systems for building organic search traffic
Cybersecurity including web security, bug bounty, SOC concepts, and malware analysis
Content creation including video editing, YouTube content, graphics, and promotional videos
I enjoy going beyond tutorials and experimenting with complete systems, from the frontend interface all the way to the backend, deployment, automation, and distribution.
Toolsura
Toolsura is one of the projects I'm most actively building.
The idea is simple: create useful, accessible online tools that solve small problems quickly.
Toolsura includes utilities for things such as text, JSON, URLs, PDFs, images, security checks, and other everyday technical tasks.
Building it has also become a practical way for me to learn how a modern web product works.
That means working on:
Product → Development → Infrastructure → SEO → Content → Distribution → Automation → Growth
Technology & Learning
I'm particularly interested in understanding how systems work rather than simply learning how to use them.
My development work has included technologies such as:
Next.js · React · TypeScript · Tailwind CSS · Go · Gin · GORM · PostgreSQL · MongoDB · Docker · RabbitMQ · MinIO · Strapi
I've also worked with Linux environments and tools such as Kali Linux, Termux, and related security tooling.
My cybersecurity interests include web application security, bug bounty research, SOC analysis, and malware analysis.
Building in Public
I also create content around the things I build and learn.
This includes:
Tutorials and technical articles
YouTube videos
Product demonstrations
Visual explainers
Promotional videos
SEO-focused educational content
My goal is to turn projects and experiments into useful resources that other people can learn from.
Why I Build
For me, building is one of the best ways to learn.
Instead of learning a technology in isolation, I prefer starting with a real problem and figuring out what is required to solve it.
Sometimes that means writing code. Other times it's debugging a deployment at midnight, or working out why a page isn't being indexed, why an API isn't behaving correctly, or how to automate something that would otherwise take hours.
Those problems are part of the process.
Every project gives me another opportunity to understand how technology works in the real world.
What I'm Working Toward
I'm continuing to develop my skills across software engineering, cybersecurity, automation, and digital products.
At the same time, I'm building Toolsura into a useful platform and experimenting with different ways to create, distribute, and grow technical products.
There is still a lot to learn, but that's exactly what makes building interesting.
Let's Connect
If you're interested in web development, cybersecurity, automation, SEO, online tools, or building products, feel free to connect.
I'm always interested in learning, experimenting, and building something useful.
Build. Learn. Improve. Repeat.