How do I submit a list of URLs?

Put one URL per line in a file, then run './gsc-indexer -creds sa.json -batch urls.txt'. You can also pipe the same list through stdin. Both methods are deduplicated with the other sources before the run starts. Comments and blank lines are ignored, so the file stays readable.

Can I inspect a whole sitemap at once?

Yes. Pass any argument ending in .xml and the tool fetches it and expands every into a URL to inspect. Nested sitemap indexes are followed all the way down, so one index can resolve to your entire URL set without you listing anything by hand.

Is there a limit on how many URLs a sitemap can produce?

Expansion is capped at 50,000 URLs as a safety limit, which is far more than most sites need. For very large properties, use a polite -delay so Google does not throttle the burst of inspection requests the tool sends on your behalf. A normal blog sitemap lands well under it.

How do positional URLs, batch, and sitemaps combine?

They are all composable and merged, then deduplicated, then run together. You can pass a couple of URLs, a -batch file, and a sitemap in the same command, and the tool will inspect the union of all three with no duplicates. Deduplication happens before any request is sent.

Should I preview a big sitemap run first?

Always. Run with -dry-run to see the full expanded URL count and contents without calling the API. At a polite -delay of 10s, a 100-URL sitemap is a 15 to 20 minute run, so confirming the set first saves you from a long run with a wrong URL list.

developer-tools

Abhay Khant

Submitting URLs in bulk, up to 50,000 per run

July 19, 2026 · 3 min read

Submit URLs to Google in bulk: batch files, stdin pipes, positional arguments, and sitemap expansion, capped at 50,000 URLs per run.

Cartoon fox with a compass as glowing document and terminal icons stream data into a machine that outputs green checkmarks

One URL at a time is fine for a quick check, but the tool is built for lists. Here are the four ways to feed it many URLs, and they all compose.

1. Batch file (`-batch`)

Put one URL per line. Lines starting with # are ignored, as are blank lines.

urls.txt:

## pages to refresh
https://www.toolsura.com/post-a/
https://www.toolsura.com/post-b/
https://www.toolsura.com/post-c/

./gsc-indexer -creds sa.json -batch urls.txt

2. Stdin pipe

Pipe a list straight in, handy in scripts:

cat urls.txt | ./gsc-indexer -creds sa.json

The tool auto-detects a piped stdin and reads from it (it only reads stdin when input isn't coming from an interactive terminal, so your keyboard won't block it).

3. Positional arguments

Multiple URLs as plain arguments also work:

./gsc-indexer -creds sa.json "https://www.toolsura.com/a/" "https://www.toolsura.com/b/"

4. Sitemaps (`.xml`)

Hand it a sitemap URL and it fetches the file and expands it into every <loc> entry:

./gsc-indexer -creds sa.json "https://www.toolsura.com/sitemap.xml" -report ./report

Sitemap indexes are followed recursively: a sitemap that lists child sitemaps (<sitemap><loc>) is expanded all the way down.
This is the easiest way to (re-)request indexing for an entire site.

`-ping-sitemap`: a deprecated nudge

The tool can ping Google's sitemap endpoint after a run (or on its own):

./gsc-indexer -creds sa.json \
  -ping-sitemap "https://www.toolsura.com/sitemap.xml" \
  "https://www.toolsura.com/blog/new-post/"

Honest caveat: Google deprecated the sitemaps ping endpoint in June 2023, and it now answers 404 ("Sitemaps ping is deprecated"). The flag still exists and the run treats a failed ping as a warning only, but don't expect it to trigger a fetch. What actually works in 2026:

Keep <lastmod> dates accurate — that's the signal Google uses when prioritizing crawls.
Submit the sitemap once in GSC (Sitemaps report); Google re-fetches on its own schedule.
Reference the sitemap in robots.txt with a Sitemap: line so any discovery path finds it.

See Google's deprecation notice.

All sources compose and dedupe

Mix them freely: the tool merges batch + args + sitemap + stdin, removes duplicates, and runs the unique set together:

./gsc-indexer -creds sa.json -batch urls.txt "https://www.toolsura.com/extra/" "https://www.toolsura.com/sitemap.xml"

The URL cap

Expansion is capped at 50,000 URLs to protect against a runaway sitemap hanging the run. If you somehow exceed it you'll get a clear error telling you so. For context, a normal blog sitemap is a few hundred URLs.

Track progress across runs: -report and -diff.
Big sitemap run stalling? Throttling and quiet mode.

Sources

Written by

Abhay Khant

Hi, I'm Abhay Thakor

I'm a developer, builder, and technology enthusiast interested in turning ideas into useful products.

I enjoy working across web development, cybersecurity, automation, SEO, and content creation. Rather than only learning technologies individually, I like putting them together to build things that people can use.

One of my main projects is Toolsura, a free online utility platform that brings practical tools into one place. I'm building Toolsura as a product and a learning project where I can explore development, infrastructure, SEO, content distribution, automation, and product growth.

What I Build

My work currently revolves around several areas:

Web development with Next.js, React, Tailwind CSS, and modern web technologies
Backend development with Go, APIs, databases, and server-side systems
Automation using n8n, APIs, Docker, and connected services
SEO and content systems for building organic search traffic
Cybersecurity including web security, bug bounty, SOC concepts, and malware analysis
Content creation including video editing, YouTube content, graphics, and promotional videos

I enjoy going beyond tutorials and experimenting with complete systems, from the frontend interface all the way to the backend, deployment, automation, and distribution.

Toolsura

Toolsura is one of the projects I'm most actively building.

The idea is simple: create useful, accessible online tools that solve small problems quickly.

Toolsura includes utilities for things such as text, JSON, URLs, PDFs, images, security checks, and other everyday technical tasks.

Building it has also become a practical way for me to learn how a modern web product works.

That means working on:

Product → Development → Infrastructure → SEO → Content → Distribution → Automation → Growth

Technology & Learning

I'm particularly interested in understanding how systems work rather than simply learning how to use them.

My development work has included technologies such as:

Next.js · React · TypeScript · Tailwind CSS · Go · Gin · GORM · PostgreSQL · MongoDB · Docker · RabbitMQ · MinIO · Strapi

I've also worked with Linux environments and tools such as Kali Linux, Termux, and related security tooling.

My cybersecurity interests include web application security, bug bounty research, SOC analysis, and malware analysis.

Building in Public

I also create content around the things I build and learn.

This includes:

Tutorials and technical articles
YouTube videos
Product demonstrations
Visual explainers
Promotional videos
SEO-focused educational content

My goal is to turn projects and experiments into useful resources that other people can learn from.

Why I Build

For me, building is one of the best ways to learn.

Instead of learning a technology in isolation, I prefer starting with a real problem and figuring out what is required to solve it.

Sometimes that means writing code. Other times it's debugging a deployment at midnight, or working out why a page isn't being indexed, why an API isn't behaving correctly, or how to automate something that would otherwise take hours.

Those problems are part of the process.

Every project gives me another opportunity to understand how technology works in the real world.

What I'm Working Toward

I'm continuing to develop my skills across software engineering, cybersecurity, automation, and digital products.

At the same time, I'm building Toolsura into a useful platform and experimenting with different ways to create, distribute, and grow technical products.

There is still a lot to learn, but that's exactly what makes building interesting.

Let's Connect

If you're interested in web development, cybersecurity, automation, SEO, online tools, or building products, feel free to connect.

I'm always interested in learning, experimenting, and building something useful.

Build. Learn. Improve. Repeat.

Frequently Asked Questions

developer-tools

Abhay Khant

Submitting URLs in bulk, up to 50,000 per run

July 19, 2026 · 3 min read

Submit URLs to Google in bulk: batch files, stdin pipes, positional arguments, and sitemap expansion, capped at 50,000 URLs per run.

One URL at a time is fine for a quick check, but the tool is built for lists. Here are the four ways to feed it many URLs, and they all compose.

1. Batch file (`-batch`)

Put one URL per line. Lines starting with # are ignored, as are blank lines.

urls.txt:

## pages to refresh
https://www.toolsura.com/post-a/
https://www.toolsura.com/post-b/
https://www.toolsura.com/post-c/

./gsc-indexer -creds sa.json -batch urls.txt

2. Stdin pipe

Pipe a list straight in, handy in scripts:

cat urls.txt | ./gsc-indexer -creds sa.json

The tool auto-detects a piped stdin and reads from it (it only reads stdin when input isn't coming from an interactive terminal, so your keyboard won't block it).

3. Positional arguments

Multiple URLs as plain arguments also work:

./gsc-indexer -creds sa.json "https://www.toolsura.com/a/" "https://www.toolsura.com/b/"

4. Sitemaps (`.xml`)

Hand it a sitemap URL and it fetches the file and expands it into every <loc> entry:

./gsc-indexer -creds sa.json "https://www.toolsura.com/sitemap.xml" -report ./report

Sitemap indexes are followed recursively: a sitemap that lists child sitemaps (<sitemap><loc>) is expanded all the way down.
This is the easiest way to (re-)request indexing for an entire site.

`-ping-sitemap`: a deprecated nudge

The tool can ping Google's sitemap endpoint after a run (or on its own):

./gsc-indexer -creds sa.json \
  -ping-sitemap "https://www.toolsura.com/sitemap.xml" \
  "https://www.toolsura.com/blog/new-post/"

Keep <lastmod> dates accurate — that's the signal Google uses when prioritizing crawls.
Submit the sitemap once in GSC (Sitemaps report); Google re-fetches on its own schedule.
Reference the sitemap in robots.txt with a Sitemap: line so any discovery path finds it.

See Google's deprecation notice.

All sources compose and dedupe

Mix them freely: the tool merges batch + args + sitemap + stdin, removes duplicates, and runs the unique set together:

./gsc-indexer -creds sa.json -batch urls.txt "https://www.toolsura.com/extra/" "https://www.toolsura.com/sitemap.xml"

The URL cap

Track progress across runs: -report and -diff.
Big sitemap run stalling? Throttling and quiet mode.

Sources

Written by

Abhay Khant

Hi, I'm Abhay Thakor

I'm a developer, builder, and technology enthusiast interested in turning ideas into useful products.

What I Build

My work currently revolves around several areas:

Web development with Next.js, React, Tailwind CSS, and modern web technologies
Backend development with Go, APIs, databases, and server-side systems
Automation using n8n, APIs, Docker, and connected services
SEO and content systems for building organic search traffic
Cybersecurity including web security, bug bounty, SOC concepts, and malware analysis
Content creation including video editing, YouTube content, graphics, and promotional videos

I enjoy going beyond tutorials and experimenting with complete systems, from the frontend interface all the way to the backend, deployment, automation, and distribution.

Toolsura

Toolsura is one of the projects I'm most actively building.

The idea is simple: create useful, accessible online tools that solve small problems quickly.

Toolsura includes utilities for things such as text, JSON, URLs, PDFs, images, security checks, and other everyday technical tasks.

Building it has also become a practical way for me to learn how a modern web product works.

That means working on:

Product → Development → Infrastructure → SEO → Content → Distribution → Automation → Growth

Technology & Learning

I'm particularly interested in understanding how systems work rather than simply learning how to use them.

My development work has included technologies such as:

Next.js · React · TypeScript · Tailwind CSS · Go · Gin · GORM · PostgreSQL · MongoDB · Docker · RabbitMQ · MinIO · Strapi

I've also worked with Linux environments and tools such as Kali Linux, Termux, and related security tooling.

My cybersecurity interests include web application security, bug bounty research, SOC analysis, and malware analysis.

Building in Public

I also create content around the things I build and learn.

This includes:

Tutorials and technical articles
YouTube videos
Product demonstrations
Visual explainers
Promotional videos
SEO-focused educational content

My goal is to turn projects and experiments into useful resources that other people can learn from.

Why I Build

For me, building is one of the best ways to learn.

Instead of learning a technology in isolation, I prefer starting with a real problem and figuring out what is required to solve it.

Those problems are part of the process.

Every project gives me another opportunity to understand how technology works in the real world.

What I'm Working Toward

I'm continuing to develop my skills across software engineering, cybersecurity, automation, and digital products.

At the same time, I'm building Toolsura into a useful platform and experimenting with different ways to create, distribute, and grow technical products.

There is still a lot to learn, but that's exactly what makes building interesting.

Let's Connect

If you're interested in web development, cybersecurity, automation, SEO, online tools, or building products, feel free to connect.

I'm always interested in learning, experimenting, and building something useful.

Build. Learn. Improve. Repeat.

Comments

Leave a Review

Rate this tool

1. Batch file (-batch)

2. Stdin pipe

3. Positional arguments

4. Sitemaps (.xml)

-ping-sitemap: a deprecated nudge

All sources compose and dedupe

The URL cap

Next

Sources

Abhay Khant

Hi, I'm Abhay Thakor

What I Build

Toolsura

Technology & Learning

Building in Public

Why I Build

What I'm Working Toward

Let's Connect

Frequently Asked Questions

How do I submit a list of URLs?

Can I inspect a whole sitemap at once?

Is there a limit on how many URLs a sitemap can produce?

How do positional URLs, batch, and sitemaps combine?

Should I preview a big sitemap run first?

1. Batch file (-batch)

2. Stdin pipe

3. Positional arguments

4. Sitemaps (.xml)

-ping-sitemap: a deprecated nudge

All sources compose and dedupe

The URL cap

Next

Sources

Abhay Khant

Hi, I'm Abhay Thakor

What I Build

Toolsura

Technology & Learning

Building in Public

Why I Build

What I'm Working Toward

Let's Connect

Frequently Asked Questions

How do I submit a list of URLs?

Can I inspect a whole sitemap at once?

Is there a limit on how many URLs a sitemap can produce?

How do positional URLs, batch, and sitemaps combine?

Should I preview a big sitemap run first?

Comments

Leave a Review

Comments

Leave a Review

1. Batch file (`-batch`)

4. Sitemaps (`.xml`)

`-ping-sitemap`: a deprecated nudge

1. Batch file (`-batch`)

4. Sitemaps (`.xml`)

`-ping-sitemap`: a deprecated nudge